Display control device, display control method, and program

ABSTRACT

A display control device includes a first acquisition unit that acquires first viewpoint position information, and a first control unit that performs a control of displaying a first viewpoint video selected from among a plurality of viewpoint videos generated based on images obtained by imaging an imaging region from a plurality of viewpoint positions on a first display unit, in which the first control unit performs a control of displaying first specific information for specifying a first viewpoint position in the first viewpoint video in a case in which the first viewpoint position indicated by the acquired first viewpoint position information is included in the first viewpoint video and performs a control of changing a display size of the first specific information depending on an angle of view of the first viewpoint video displayed on the first display unit.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of InternationalApplication No. PCT/JP2020/024637, filed Jun. 23, 2020, the disclosureof which is incorporated herein by reference in its entirety. Further,this application claims priority from Japanese Patent Application No.2019-122033, filed Jun. 28, 2019, the disclosure of which isincorporated by reference herein.

BACKGROUND 1. Technical Field

The technology of the present disclosure relates to a display controldevice, a display control method, and a program.

2. Related Art

JP2018-190336A discloses a method executed by a computer for providing avirtual space by a head mount device, the method comprising a step ofdefining the virtual space, a step of receiving a first voice signalcorresponding utterance of a user of the head mount device, and a stepof executing imaging in the virtual space by using the first voicesignal as a trigger. The method disclosed in JPA2018-190336A furthercomprises a step of disposing a first avatar object corresponding to theuser in the virtual space and a step of disposing a second avatar objectcorresponding to a user of another computer capable of communicatingwith the computer in the virtual space, in which executing imagingincludes imaging at least a part of each of the first and second avatarobjects based on positional information of the first and second avatarobjects.

JP2018-106297A discloses a mixed reality presentation system. The mixedreality presentation system disclosed in JP2018-106297A includes adevice having an imaging unit and a display unit, the device being aplurality of head mountable devices, a storage unit that storesidentification information of a local coordinate space to which eachdevice belongs, a CG model information for drawing a CG model to becomposed on an image captured by the imaging unit, and avatarinformation for drawing an avatar displayed on behalf of a person withexperience who mounts each device, a detection unit that detects aposition and a posture of each device, and a control unit that composesthe CG model and the avatar on the image captured by the imaging unit ofeach device based on the position and the posture of the device andoutputs the composite image obtained by composing to each display unit.

In the mixed reality presentation system disclosed in JP2018-106297A,the control unit includes a composing unit that, in a case in which oneof the plurality of devices is used as an interest device, composes theCG model based on the position and the posture of the interest deviceand an avatar corresponding to another device belonging to the localcoordinate space, which is different from the interest device, on theimage captured by the imaging unit of the interest device, and thecomposing unit composes the CG model and the avatar by controlling anoutput of the avatar corresponding the other device such that the CGmodel is visually recognized based on a positional relationship betweenthe interest device, the CG model, and the other device.

SUMMARY

One embodiment according to the technology of the present disclosureprovides a display control device, a display control method, and aprogram which can, in a state in which presence of a specific person canbe perceived through a viewpoint video selected from among a pluralityof viewpoint videos, change a sense of presence of the specific persondepending on an angle of view of the viewpoint video visuallyrecognized.

A first aspect of the technology of the present disclosure relates to adisplay control device including a first acquisition unit that acquiresfirst viewpoint position information indicating a first viewpointposition of a first person with respect to an imaging region, and afirst control unit that performs a control of displaying a firstviewpoint video selected from among a plurality of viewpoint videosgenerated based on images obtained by imaging the imaging region from aplurality of viewpoint positions different from each other on a firstdisplay unit, the first display unit being capable of displaying a videovisually recognized by a second person different from the first person,in which the first control unit performs a control of displaying firstspecific information for specifying the first viewpoint positionindicated by the first viewpoint position information acquired by thefirst acquisition unit in the first viewpoint video and performs acontrol of changing a display size of the first specific informationdepending on an angle of view of the first viewpoint video displayed onthe first display unit in a case in which the first viewpoint positionindicated by the first viewpoint position information acquired by thefirst acquisition unit is included in the first viewpoint video.

A second aspect of the technology of the present disclosure relates tothe display control device according to the first aspect, in which thefirst control unit performs a control of causing the first display unitto set a degree of difference between an image quality of the firstviewpoint video and an image quality of the first specific informationwithin a first predetermined range.

A third aspect of the technology of the present disclosure relates tothe display control device according to the first aspect or the secondaspect, in which the first control unit performs a control of causingthe first display unit to change a display aspect of the first specificinformation depending on a relationship between a display size of thefirst viewpoint video and the display size of the first specificinformation.

A fourth aspect of the technology of the present disclosure relates tothe display control device according to the third aspect, in which thefirst control unit changes the display aspect of the first specificinformation by causing the first display unit to hide the first specificinformation, to display only an outline of the first specificinformation, or to translucently display the first specific informationin a case in which a ratio of the display size of the first specificinformation to the display size of the first viewpoint video is equal toor more than a first threshold value.

A fifth aspect of the technology of the present disclosure relates tothe display control device according to the third aspect or the fourthaspect, in which the first control unit changes the display aspect ofthe first specific information depending on the relationship between thedisplay size of the first viewpoint video and the display size of thefirst specific information and a relationship between a display positionof the first viewpoint video and a display position of the firstspecific information.

A sixth aspect of the technology of the present disclosure relates tothe display control device according to the third aspect, in which thefirst control unit causes the first display unit to display the firstspecific information in a display aspect emphasized more than otherregions in the first viewpoint video in a case in which a ratio of thedisplay size of the first specific information to the display size ofthe first viewpoint video is less than a second threshold value.

A seventh aspect of the technology of the present disclosure relates tothe display control device according to any one of the first aspect tothe sixth aspect, in which the first display unit is provided in a firsthead mounted display mounted on the second person.

An eighth aspect of the technology of the present disclosure relates tothe display control device according to any one of the first aspect tothe seventh aspect, in which the first viewpoint video is a viewpointvideo selected from among the plurality of viewpoint videos in responseto a first instruction for selecting any one of the plurality ofviewpoint videos, which is received by a first reception unit capable ofreceiving the first instruction.

A ninth aspect of the technology of the present disclosure relates tothe display control device according to any one of the first aspect tothe eighth aspect, in which the first acquisition unit further acquiresfirst visual line direction information indicating a first visual linedirection of the first person with respect to the imaging region, andthe first specific information includes information for specifying thefirst visual line direction indicated by the first visual line directioninformation acquired by the first acquisition unit.

A tenth aspect of the technology of the present disclosure relates tothe display control device according to any one of the first aspect tothe ninth aspect, in which each of the plurality of viewpoint videos haspeculiar viewpoint position information indicating a peculiar viewpointposition, each of the plurality of viewpoint videos is a video showingthe imaging region observed from the corresponding peculiar viewpointposition, and the first viewpoint position information is the peculiarviewpoint position information of any one of the plurality of viewpointvideos.

An eleventh aspect of the technology of the present disclosure relatesto the display control device according to the tenth aspect, in whichthe first acquisition unit acquires the peculiar viewpoint positioninformation corresponding to a second viewpoint video selected fromamong the plurality of viewpoint videos which is displayed on a seconddisplay unit, the second display unit being capable of displaying avideo visually recognized by the first person and the second viewpointvideo, as the first viewpoint position information.

A twelfth aspect of the technology of the present disclosure relates tothe display control device according to any one of the first aspect tothe eighth aspect, in which the first acquisition unit further acquiresfirst visual line direction information indicating a first visual linedirection of the first person with respect to the imaging region, thefirst specific information includes information for specifying the firstvisual line direction indicated by the first visual line directioninformation acquired by the first acquisition unit, and the firstacquisition unit acquires information indicating a direction facing asecond viewpoint video selected from among the plurality of viewpointvideos which is displayed on a second display unit, the second displayunit being capable of displaying a video visually recognized by thefirst person and the second viewpoint video, as the first visual linedirection information.

A thirteenth aspect of the technology of the present disclosure relatesto the display control device according to the twelfth aspect, in whichthe second display unit is provided in a second head mounted displaymounted on the first person.

A fourteenth aspect of the technology of the present disclosure relatesto the display control device according to any one of the eleventhaspect to the thirteenth aspect, in which the second viewpoint video isa viewpoint video selected from among the plurality of viewpoint videosin response to a second instruction for selecting any one of theplurality of viewpoint videos, which is received by a second receptionunit capable of receiving the second instruction.

A fifteenth aspect of the technology of the present disclosure relatesto the display control device according any one of the first aspect tothe eighth aspect further including a second acquisition unit thatacquires second viewpoint position information indicating a secondviewpoint position of the second person with respect to the imagingregion, and a second control unit that performs a control of displayinga second viewpoint video selected from among the plurality of viewpointvideos on a second display unit, the second display unit being capableof displaying a video visually recognized by the first person, in whichthe second control unit performs a control of displaying second specificinformation for specifying the second viewpoint position indicated bythe second viewpoint position information acquired by the secondacquisition unit in the second viewpoint video and performs a control ofchanging a display size of the second specific information depending onan angle of view of the second viewpoint video displayed on the seconddisplay unit in a case in which the second viewpoint position indicatedby the second viewpoint position information acquired by the secondacquisition unit is included in the second viewpoint video.

A sixteenth aspect of the technology of the present disclosure relatesto the display control device according to the fifteenth aspect, inwhich the second control unit performs a control of causing the seconddisplay unit to set a degree of difference between an image quality ofthe second viewpoint video and an image quality of the second specificinformation within a second predetermined range.

A seventeenth aspect of the technology of the present disclosure relatesto the display control device according to the fifteenth aspect or thesixteenth aspect, in which the first control unit performs a control ofcausing the second display unit to change a display aspect of the secondspecific information depending on a relationship between a display sizeof the second viewpoint video and the display size of the secondspecific information.

An eighteenth aspect of the technology of the present disclosure relatesto the display control device according to the seventeenth aspect, inwhich the second control unit changes the display aspect of the secondspecific information by causing the second display unit to hide thesecond specific information, to display only an outline of the secondspecific information, or to translucently display the second specificinformation in a case in which a ratio of the display size of the secondspecific information to the display size of the second viewpoint videois equal to or more than a third threshold value.

A nineteenth aspect of the technology of the present disclosure relatesto the display control device according to the fifteenth aspect or thesixteenth aspect, in which the second control unit changes the displayaspect of the second specific information depending on a relationshipbetween a display size of the second viewpoint video and the displaysize of the second specific information and a relationship between adisplay position of the second viewpoint video and a display position ofthe second specific information.

A twentieth aspect of the technology of the present disclosure relatesto the display control device according to the seventeenth aspect, inwhich the second control unit causes the second display unit to displaythe second specific information in a display aspect emphasized more thanother regions in the second viewpoint video in a case in which a ratioof the display size of the second specific information to the displaysize of the second viewpoint video is less than a fourth thresholdvalue.

A twenty-first aspect of the technology of the present disclosurerelates to the display control device according to any one of the firstaspect to the twentieth aspect, in which the second acquisition unitfurther acquires second visual line direction information indicating asecond visual line direction of the second person with respect to theimaging region, and the second specific information includes informationfor specifying the second visual line direction indicated by the secondvisual line direction information acquired by the second acquisitionunit.

A twenty-second aspect of the technology of the present disclosurerelates to the display control device according to any one of thefifteenth aspect to the twenty-first aspect, in which each of theplurality of viewpoint videos has peculiar viewpoint positioninformation indicating a peculiar viewpoint position, each of theplurality of viewpoint videos is a video showing the imaging regionobserved from the corresponding peculiar viewpoint position, and each ofthe first viewpoint position information and the second viewpointposition information is the peculiar viewpoint position information ofany one of the plurality of viewpoint videos.

A twenty-third aspect of the technology of the present disclosurerelates to the display control device according to the twenty-secondaspect, in which the first acquisition unit acquires the peculiarviewpoint position information corresponding to the second viewpointvideo selected from among the plurality of viewpoint videos which isdisplayed on the second display unit, the second display unit beingcapable of displaying the video visually recognized by the first personand the second viewpoint video, as the first viewpoint positioninformation, and the second acquisition unit acquires the peculiarviewpoint position information corresponding to the first viewpointvideo displayed on the first display unit as the second viewpointposition information.

A twenty-fourth aspect of the technology of the present disclosurerelates to the display control device according to any one of thefifteenth aspect to the twenty-third aspect, in which the firstacquisition unit further acquires first visual line directioninformation indicating a first visual line direction of the first personwith respect to the imaging region, the first specific informationincludes information for specifying the first visual line directionindicated by the first visual line direction information acquired by thefirst acquisition unit, the first acquisition unit acquires informationindicating a direction facing the second viewpoint video displayed onthe second display unit as the first visual line direction information,the second acquisition unit further acquires second visual linedirection information indicating a second visual line direction of thesecond person with respect to the imaging region, the second specificinformation includes information for specifying the second visual linedirection indicated by the second visual line direction informationacquired by the second acquisition unit, and the second acquisition unitacquires information indicating a direction facing the first viewpointvideo displayed on the first display unit as the second visual linedirection information.

A twenty-fifth aspect of the technology of the present disclosurerelates to the display control device according to any one of thefifteenth aspect to the twenty-fourth aspect, in which the seconddisplay unit is provided in a second head mounted display mounted on thefirst person.

A twenty-sixth aspect of the technology of the present disclosurerelates to the display control device according to any one of thefifteenth aspect to the twenty-fifth aspect, in which the secondviewpoint video is a viewpoint video selected from among the pluralityof viewpoint videos in response to a second instruction for selectingany one of the plurality of viewpoint videos, which is received by asecond reception unit capable of receiving the second instruction.

A twenty-seventh aspect of the technology of the present disclosurerelates to the display control device according to any one of thefifteenth aspect to the twenty-sixth aspect further including a firstsetting unit that performs setting for hiding the second specificinformation in a case in which a first predetermined condition issatisfied.

A twenty-eighth aspect of the technology of the present disclosurerelates to the display control device according to any one of the firstaspect to the twenty-seventh aspect, in which a viewpoint position of atleast one of the first person or the second person with respect to theimaging region is limited to a partial region of the imaging region.

A twenty-ninth aspect of the technology of the present disclosurerelates to the display control device according to any one of the firstaspect to the twenty-eighth aspect further including a second settingunit that performs setting for hiding the first specific information ina case in which a second predetermined condition is satisfied.

A thirtieth aspect of the technology of the present disclosure relatesto the display control device according to any one of the first aspectto the twenty-ninth aspect, in which at least one of the plurality ofviewpoint videos is a virtual viewpoint video.

A thirty-first aspect of the technology of the present disclosurerelates to a display control method including acquiring first viewpointposition information indicating a first viewpoint position of a firstperson with respect to an imaging region, performing a control ofdisplaying a first viewpoint video selected from among a plurality ofviewpoint videos generated based on images obtained by imaging theimaging region from a plurality of viewpoint positions different fromeach other on a first display unit, the first display unit being capableof displaying a video visually recognized by a second person differentfrom the first person, and performing a control of displaying firstspecific information for specifying the first viewpoint positionindicated by the acquired first viewpoint position information in thefirst viewpoint video and performing a control of changing a displaysize of the first specific information depending on an angle of view ofthe first viewpoint video displayed on the first display unit in a casein which the first viewpoint position indicated by the acquired firstviewpoint position information is included in the first viewpoint video.

A thirty-second aspect of the technology of the present disclosurerelates to a program causing a computer to execute a process includingacquiring first viewpoint position information indicating a firstviewpoint position of a first person with respect to an imaging region,performing a control of displaying a first viewpoint video selected fromamong a plurality of viewpoint videos generated based on images obtainedby imaging the imaging region from a plurality of viewpoint positionsdifferent from each other on a first display unit, the first displayunit being capable of displaying a video visually recognized by a secondperson different from the first person, and performing a control ofdisplaying first specific information for specifying the first viewpointposition indicated by the acquired first viewpoint position informationin the first viewpoint video and performing a control of changing adisplay size of the first specific information depending on an angle ofview of the first viewpoint video displayed on the first display unit ina case in which the first viewpoint position indicated by the acquiredfirst viewpoint position information is included in the first viewpointvideo.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments of the technology of the disclosure will bedescribed in detail based on the following figures, wherein:

FIG. 1 is a schematic perspective diagram showing an example of anexternal configuration of an information processing system according toan embodiment;

FIG. 2 is a schematic perspective diagram showing an example of anexternal configuration of a first HMD and a second HMD provided in theinformation processing system according to the embodiment;

FIG. 3 is a conceptual diagram showing an example of a relationshipbetween a display control device provided in the information processingsystem according to the embodiment and peripheral device thereof;

FIG. 4 is a block diagram showing an example of a hardware configurationof an electric system of the display control device according to theembodiment;

FIG. 5 is a block diagram showing an example of a hardware configurationof an electric system of each of a first smartphone and a secondsmartphone according to the embodiment;

FIG. 6 is a block diagram showing an example of a hardware configurationof an electric system of each of the first HMD and the second HMDaccording to the embodiment;

FIG. 7 is a block diagram showing an example of a main function of thedisplay control device according to the embodiment;

FIG. 8 is a conceptual diagram provided for describing an example of aprocess content of a viewpoint video generation process executed by aCPU of the display control device according to the embodiment;

FIG. 9 is a block diagram showing each example of a viewpoint video, aviewpoint video identifier, peculiar viewpoint position information,peculiar visual line direction information, and peculiar angle-of-viewinformation generated by executing the viewpoint video generationprocess according to the embodiment;

FIG. 10 is a conceptual diagram showing an example of an aspect in whicha first viewpoint visual line instruction is given to the firstsmartphone according to the embodiment and an example of an aspect inwhich a second viewpoint visual line instruction is given to the secondsmartphone according to the embodiment;

FIG. 11 is a conceptual diagram showing an example of an aspect in whichthe first viewpoint visual line instruction is transmitted from thefirst smartphone according to the embodiment to the display controldevice and the second viewpoint visual line instruction is transmittedfrom the second smartphone according to the embodiment to the displaycontrol device;

FIG. 12 is a block diagram showing an example of specific functions of afirst control unit and a second control unit of the display controldevice according to the embodiment;

FIG. 13 is a block diagram showing an example of an aspect in which theviewpoint video generated by executing the viewpoint video generationprocess by the CPU of the display control device according to theembodiment is acquired by a first viewpoint video acquisition unit and asecond viewpoint video acquisition unit, and the viewpoint videoidentifier, the peculiar viewpoint position information, the peculiarvisual line direction information, and the peculiar angle-of-viewinformation are stored in a memory;

FIG. 14 is a block diagram showing an example of each process content ofa first acquisition unit, the first viewpoint video acquisition unit,and a first determination unit according to the embodiment;

FIG. 15 is a conceptual diagram showing an example of an aspect in whicha first viewpoint position is included in a first viewpoint videoselected from among a plurality of viewpoint videos;

FIG. 16 is a block diagram showing an example of each process content ofa second acquisition unit, the second viewpoint video acquisition unit,and a second determination unit according to the embodiment;

FIG. 17 is a conceptual diagram showing an example of an aspect in whicha second viewpoint position is included in a second viewpoint videoselected from among the plurality of viewpoint videos;

FIG. 18 is a block diagram provided for describing an example of eachprocess content of a first composing unit and a first viewpoint videooutput unit according to the embodiment;

FIG. 19 is a conceptual diagram showing an example of a firstavatar-containing viewpoint video generated by the first composing unitaccording to the embodiment;

FIG. 20 is a block diagram provided for describing an example of eachprocess content of a second composing unit and a second viewpoint videooutput unit according to the embodiment;

FIG. 21 is a block diagram provided for describing an example of aprocess content of a first avatar display size changing unit accordingto the embodiment;

FIG. 22 is a conceptual diagram showing an example of an aspect in whicha display aspect of a first avatar is changed by the first avatardisplay size changing unit according to the embodiment;

FIG. 23 is a block diagram provided for describing an example of aprocess content of a second avatar display size changing unit accordingto the embodiment;

FIG. 24 is a block diagram showing an example of a relationship betweenthe first avatar display size changing unit and a first image qualitycontrol unit according to the embodiment;

FIG. 25 is a conceptual diagram showing an example of an aspect of thefirst avatar-containing viewpoint video in a case in which an imagequality of the first avatar is controlled by the first image qualitycontrol unit according to the embodiment;

FIG. 26 is a block diagram showing an example of a relationship betweenthe second avatar display size changing unit and a second image qualitycontrol unit according to the embodiment;

FIG. 27 is a block diagram showing an example of a relationship betweenthe first image quality control unit, a first display aspect changingunit, and the first viewpoint video output unit according to theembodiment;

FIG. 28 is a conceptual diagram showing an example of a relationshipbetween a size of the first avatar of which the display aspect is to bechanged by the first display aspect changing unit according to theembodiment and a size of the first viewpoint video;

FIG. 29 is a conceptual diagram showing an example of an aspect in whichthe first avatar is hidden from the first avatar-containing viewpointvideo shown in FIG. 28;

FIG. 30 is a block diagram showing an example of a relationship betweenthe second image quality control unit, a second display aspect changingunit, and the second viewpoint video output unit according to theembodiment;

FIG. 31 is a block diagram showing an example of a relationship betweenthe first display aspect changing unit, the first viewpoint video outputunit, and the first HMD according to the embodiment;

FIG. 32 is a block diagram showing an example of a relationship betweenthe second display aspect changing unit, the second viewpoint videooutput unit, and the second HMD according to the embodiment;

FIG. 33 is a conceptual diagram showing an example of an aspect in whichan avatar hide instruction is transmitted from the first smartphoneaccording to the embodiment to a setting unit of the display controldevice, and an avatar hide instruction is transmitted from the secondsmartphone according to the embodiment to the setting unit of thedisplay control device;

FIG. 34 is a block diagram provided for describing an example of aprocess content of the setting unit according to the embodiment;

FIG. 35 is a flowchart showing an example of a flow of a first displaycontrol process according to the embodiment;

FIG. 36 is a continuation of the flowchart shown in FIG. 35;

FIG. 37 is a continuation of the flowchart shown in FIGS. 35 and 36;

FIG. 38 is a flowchart showing an example of a flow of a second displaycontrol process according to the embodiment;

FIG. 39 is a continuation of the flowchart shown in FIG. 38;

FIG. 40 is a continuation of the flowchart shown in FIGS. 38 and 39;

FIG. 41 is a flowchart showing an example of a flow of a setting processaccording to the embodiment;

FIG. 42 is a conceptual diagram showing an example of an aspect in whichthe first avatar is superimposed on a center region of the firstavatar-containing viewpoint video;

FIG. 43 is a conceptual diagram showing an example of an aspect in whichthe first avatar is superimposed on a region-of-interest in the firstavatar-containing viewpoint video;

FIG. 44 is a conceptual diagram showing an example of an aspect in whichthe first avatar is superimposed on the region-of-interest in the firstavatar-containing viewpoint video and the first avatar is translucent;

FIG. 45 is a block diagram showing a modification example of aconfiguration of the first HMD according to the embodiment;

FIG. 46 is a block diagram showing a modification example of aconfiguration of the second HMD according to the embodiment; and

FIG. 47 is a block diagram showing an example of an aspect in which adisplay control device program in a storage medium in which a displaycontrol device program according to the embodiment is stored isinstalled in a computer of the display control device.

DETAILED DESCRIPTION

An example of an embodiment according to the technology of the presentdisclosure will be described with reference to the accompanyingdrawings.

First, the terms used in the following description will be described.

CPU refers to an abbreviation of “central processing unit”. RAM refersto an abbreviation of “random access memory”. DRAM refers to anabbreviation of “dynamic random access memory”. SRAM refers to anabbreviation of “static random access memory”. ROM refers to anabbreviation of “read only memory”. SSD refers to an abbreviation of“solid state drive”. HDD refers to an abbreviation of “hard disk drive”.EEPROM refers to an abbreviation of “electrically erasable andprogrammable read only memory”. OF refers to an abbreviation of“interface”. IC refers to an abbreviation of “integrated circuit”. ASICrefers to an abbreviation of “application specific integrated circuit”.PLD refers to an abbreviation of “programmable logic device”. FPGArefers to an abbreviation of “field-programmable gate array”. SoC refersto an abbreviation of “system-on-a-chip”. CMOS refers to an abbreviationof “complementary metal oxide semiconductor”. CCD refers to anabbreviation of “charge coupled device”. EL refers to an abbreviation of“electro-luminescence”. GPU refers to an abbreviation of “graphicsprocessing unit”. LAN refers to an abbreviation of “local area network”.3D refers to an abbreviation of “3 dimension”. USB refers to anabbreviation of “universal serial bus”. HMD refers to an abbreviation of“head mounted display”. fps refers to an abbreviation of “frame persecond”. GPS refers to an abbreviation of “global positioning system”.

For example, as shown in FIG. 1, an information processing system 10comprises a display control device 12, a first smartphone 14A, a secondsmartphone 14B, a plurality of imaging apparatuses 16, an imagingapparatus 18, and a wireless communication base station (hereinafter,simply referred to as “base station”) 20, a first HMD 34A, and a secondHMD 34B.

The imaging apparatuses 16 and 18 are devices for imaging having a CMOSimage sensor, and each have an optical zoom function and a digital zoomfunction. Note that another type of image sensor, such as a CCD imagesensor, may be adopted instead of the CMOS image sensor. Hereinafter,for convenience of description, in a case in which a distinction is notnecessary, the imaging apparatus 18 and the plurality of imagingapparatuses 16, are referred to as “plurality of imaging apparatuses”without reference numeral.

The plurality of imaging apparatuses 16 are installed in a soccerstadium 22. Each of the plurality of imaging apparatuses 16 is disposedso as to surround a soccer field 24, and images a region including thesoccer field 24 as an imaging region. Here, an aspect example isdescribed in which each of the plurality of imaging apparatuses 16 isdisposed so as to surround the soccer field 24. However, the technologyof the present disclosure is not limited to this, and the disposition ofthe plurality of imaging apparatuses 16 is decided depending on avirtual viewpoint image to be generated. The plurality of imagingapparatuses 16 may be disposed so as to surround the whole soccer field24, or the plurality of imaging apparatuses 16 may be disposed so as tosurround a specific part thereof. The imaging apparatus 18 is installedin an unmanned aerial vehicle (for example, a drone), and images theregion including the soccer field 24 as the imaging region in abird's-eye view from the sky. The imaging region of the region includingthe soccer field 24 in a bird's-eye view from the sky refers to animaging face on the soccer field 24 by the imaging apparatus 18.

The display control device 12 is installed in a control room 32. Theplurality of imaging apparatuses 16 and the display control device 12are connected to each other via a LAN cable 30, and the display controldevice 12 controls the plurality of imaging apparatuses 16 and acquiresan image obtained by being imaged by each of the plurality of imagingapparatuses 16. Note that although the connection using a wiredcommunication method by the LAN cable 30 is described as an examplehere, the technology of the present disclosure is not limited to this,and the connection using a wireless communication method may be used.

The base station 20 transmits and receives various pieces of informationto and from the display control device 12, the first smartphone 14A, thesecond smartphone 14B, the first HMD 34A, the second HMD 34B, and anunmanned aerial vehicle 27 via radio waves. That is, the display controldevice 12 is connected to the first smartphone 14A, the secondsmartphone 14B, the first HMD 34A, the second HMD 34B, and the unmannedaerial vehicle 27 via the base station 20 in a wirelessly communicablemanner. The display control device 12 controls the unmanned aerialvehicle 27 by wirelessly communicating with the unmanned aerial vehicle27 via the base station 20, and acquires the image obtained by beingimaged by the imaging apparatus 18 from the unmanned aerial vehicle 27.

The display control device 12 is a device corresponding to a server, andthe first smartphone 14A, the second smartphone 14B, the first HMD 34A,and the second HMD 34B are devices corresponding to a client terminalwith respect to the display control device 12. Note that, in thefollowing, in a case in which a distinction is not necessary, the firstsmartphone 14A, the second smartphone 14B, the first HMD 34A, and thesecond HMD 34B are referred to as “terminal device” without referencenumeral.

The display control device 12 and the terminal device wirelesslycommunicate with each other via the base station 20, so that theterminal device requests the display control device 12 to providevarious services, and the display control device 12 provides theservices to the terminal device in response to the request from theterminal device.

The display control device 12 acquires a plurality of the images fromthe plurality of imaging apparatuses, and transmits a video generatedbased on the acquired plurality of images to the terminal device via thebase station 20. In the example shown in FIG. 1, a viewer 28A owns thefirst smartphone 14A, and the first HMD 34A is mounted on a head of theviewer 28A.

A viewer 28B is a different person from the viewer 28A. The viewer 28Bowns the second smartphone 14B, and the second HMD 34B is mounted on ahead of the viewer 28B. The video transmitted from the display controldevice 12 (hereinafter, also referred to as “distribution video”) isreceived by the terminal device, and the distribution video received bythe terminal device is visually recognized by the viewers 28A and 28Bthrough the terminal device. Note that the viewer 28A is an example of a“second person” according to the technology of the present disclosure,and the viewer 28B is an example of a “first person” according to thetechnology of the present disclosure. In addition, the distributionvideo is an example of “video” according to the technology of thepresent disclosure.

For example, as shown in FIG. 2, the first HMD 34A comprises a body part11A and a mounting part 13A. In a case in which the first HMD 34A ismounted on the viewer 28A, the body part 11A is positioned in front ofthe viewer 28A, and the mounting part 13A is positioned in the upperhalf of the head of the viewer 28A.

The mounting part 13A is a band-shaped member having a width of aboutseveral centimeters, and comprises an inner ring 13A1 and an outer ring15A1. The inner ring 13A1 is formed in an annular shape and is fixed ina state of being closely attached to the upper half of the head of theviewer 28A. The outer ring 15A1 is formed in a shape in which anoccipital side of the viewer 28A is cut out. The outer ring 15A1 bendsoutward from an initial position or shrinks inward from a bent statetoward the initial position depending on adjustment of a size of theinner ring 13A1.

The body part 11A comprises a protective frame 11A1, a computer 150, anda display 156. The computer 150 controls the whole first HMD 34A. Theprotective frame 11A1 is one transparent plate curved so as to cover thewhole eyes of the viewer 28A, and is made of, for example, transparentcolored plastic.

The display 156 comprises a screen 156A and a projection unit 156B, andthe projection unit 156B is controlled by the computer 150. The screen156A is disposed inside the protective frame 11A1. The screen 156A isassigned to each of the eyes of viewer 28A. The screen 156A is made of atransparent material similar to the protective frame 11A1. The viewer28A visually recognizes a real space via the screen 156A and theprotective frame 11A1 with the naked eye. That is, the first HMD 34A isa transmission type HMD.

The screen 156A is located at a position facing the eyes of the viewer28A, and the distribution video is projected on an inner surface of thescreen 156A (surface on the viewer 28A side) by the projection unit 156Bunder the control of the computer 150. Since the projection unit 156B isa well-known device, detailed description thereof will be omitted.However, the projection unit 156B is a device including a displayelement, such as a liquid crystal, which displays the distribution videoand projection optical system that projects the distribution videodisplayed on the display element toward the inner surface of the screen156A. The screen 156A is realized by using a half mirror that reflectsthe distribution video projected by the projection unit 156B andtransmits the light in the real space. The projection unit 156B projectsthe distribution video on the inner surface of the screen 156A at apredetermined frame rate (for example, 60 fps). The distribution videois reflected by the inner surface of the screen 156A and is incident onthe eyes of the viewer 28A. As a result, the viewer 28A visuallyrecognizes the distribution video. Note that the half mirror isdescribed as an example of the screen 156A here, but the technology ofthe present disclosure is not limited to this, and the screen 156Aitself may be used as the display element, such as the liquid crystal.

Note that the second HMD 34B also has the same configuration as thefirst HMD 34A, and the first HMD 34A is applied to the viewer 28A,whereas the second HMD 34B is applied to the viewer 28B.

The second HMD 34B comprises a body part 11B and a mounting part 13B.The mounting part 13B corresponds to the mounting part 13A of the firstHMD 34A. In the example shown in FIG. 2, an inner ring 13B1 correspondsto the inner ring 13A1 and an outer ring 15B1 corresponds to the outerring 15A1. In addition, in the example shown in FIG. 2, the body part11B corresponds to the body part 11A of the first HMD 34A. In addition,in the example shown in FIG. 2, a protective frame 11B1 corresponds tothe protective frame 11A1, a display 206 corresponds to the display 156,a computer 200 corresponds to the computer 150. In addition, in theexample shown in FIG. 2, a screen 206A corresponds to the screen 156A,and a projection unit 206B corresponds to the projection unit 156B.

For example, as shown in FIG. 3, the display control device 12 acquiresa bird's-eye view video 46A showing the region including the soccerfield 24 in a case of being observed from the sky from the unmannedaerial vehicle 27. The bird's-eye view video 46A is a moving imageobtained by imaging the region including the soccer field 24 as theimaging region (hereinafter, also simply referred to as “imagingregion”) in a bird's-eye view from the sky by the imaging apparatus 18of the unmanned aerial vehicle 27. Note that although the bird's-eyeview video 46A is described as an example here, the technology of thepresent disclosure is not limited to this, and may be a still imageshowing the region including the soccer field 24 in a case of beingobserved from the sky.

The display control device 12 acquires an imaging video 46B showing theimaging region in a case of being observed from each position of theplurality of imaging apparatuses 16 from each of the plurality ofimaging apparatuses 16. The imaging video 46B is a moving image obtainedby imaging the imaging region by each of the plurality of imagingapparatuses 16. Note that although the imaging video 46B is described asan example here, the technology of the present disclosure is not limitedto this, and may be a still image showing the imaging region in a caseof being observed from each position of the plurality of imagingapparatuses 16.

The bird's-eye view video 46A and the imaging video 46B are videosobtained by imaging the images from a plurality of viewpoint positionsin which the regions including the soccer field 24 are different fromeach other, and are examples of “image” according to the technology ofthe present disclosure.

The display control device 12 generates a virtual viewpoint video 46Cbased on the bird's-eye view video 46A and the imaging video 46B. Thevirtual viewpoint video 46C is video showing the imaging region in acase in which the imaging region is observed from a viewpoint positionand a visual line direction different from a viewpoint position and avisual line direction of each of the plurality of imaging apparatuses.In the example shown in FIG. 3, the virtual viewpoint video 46C refersto the virtual viewpoint video showing the imaging region in a case inwhich the imaging region is observed from a viewpoint position 42 and avisual line direction 44 in a spectator seat 26. Examples of the virtualviewpoint video 46C include a moving image using a 3D polygon. Note thatthe moving image is described as an example of the virtual viewpointvideo 46C here, but the technology of the present disclosure is notlimited to this, and a still image using the 3D polygon may be used.Here, an aspect example is described in which the bird's-eye view video46A obtained by being imaged by the imaging apparatus 18 is alsoprovided for generation, but the technology of the present disclosure isnot limited to this. For example, the bird's-eye view video 46A is notprovided for generation of the virtual viewpoint video 46C, and only aplurality of the imaging videos 46B obtained by being imaged by theplurality of imaging apparatuses 16 may be provided for generation ofthe virtual viewpoint video 46C. That is, the virtual viewpoint video46C may be generated only from the videos obtained by being imaged bythe plurality of imaging apparatuses 16 without using the video obtainedby the imaging apparatus 18 (for example, a drone). In addition, in acase in which the video obtained from the imaging apparatus 18 (forexample, a drone) is used, a more accurate virtual viewpoint video canbe generated.

The display control device 12 selectively transmits the bird's-eye viewvideo 46A, the imaging video 46B, and the virtual viewpoint video 46C asthe distribution video to the terminal device.

For example, as shown in FIG. 4, the display control device 12 comprisesa computer 50, a reception device 52, a display 53, a firstcommunication I/F 54, and a second communication I/F 56. The computer 50comprises a CPU 58, a storage 60, and a memory 62, and the CPU 58, thestorage 60, and the memory 62 are connected to each other via a bus line64. In the example shown in FIG. 4, for convenience of illustration, onebus line is shown as the bus line 64, but a data bus, an address bus, acontrol bus, and the like are included in the bus line 64.

The CPU 58 controls the whole display control device 12. Variousparameters and various programs are stored in the storage 60. Thestorage 60 is a non-volatile storage device. Here, an EEPROM is adoptedas an example of the storage 60, but the technology of the presentdisclosure is not limited to this, and a mask ROM, an HDD, an SSD, orthe like may be used. The memory 62 is a volatile storage device.Various pieces of information are transitorily stored in the memory 62.The memory 62 is used as a work memory by the CPU 58. Here, a DRAM isadopted as an example of the memory 62, but the technology of thepresent disclosure is not limited to this, and another type of volatilestorage device, such as an SRAM, may be used.

The reception device 52 receives the instruction from a user or the likeof the display control device 12. Examples of the reception device 52include a touch panel, a hard key, and a mouse. The reception device 52is connected to the bus line 64, and the CPU 58 acquires the instructionreceived by the reception device 52.

The display 53 is connected to the bus line 64 and displays variouspieces of information under the control of the CPU 58. Examples of thedisplay 53 include a liquid crystal display. Note that another type ofdisplay, such as an organic EL display, may be adopted as the display 53without being limited to the liquid crystal display.

The first communication I/F 54 is connected to the LAN cable 30. Thefirst communication I/F 54 is realized by a device having an FPGA, forexample. The first communication I/F 54 is connected to the bus line 64and controls the exchange of various pieces of information between theCPU 58 and the plurality of imaging apparatuses 16. For example, thefirst communication I/F 54 controls the plurality of imaging apparatuses16 in response to the request of the CPU 58. In addition, the firstcommunication I/F 54 acquires the imaging video 46B (see FIG. 3)obtained by being imaged by each of the plurality of imaging apparatuses16, and outputs the acquired imaging video 46B to the CPU 58.

The second communication I/F 56 is connected to the base station 20 inthe wirelessly communicable manner. The second communication I/F 56 isrealized by a device having an FPGA, for example. The secondcommunication I/F 56 is connected to the bus line 64. The secondcommunication I/F 56 controls the exchange of various pieces ofinformation between the CPU 58 and the unmanned aerial vehicle 27 by thewireless communication method via the base station 20. In addition, thesecond communication I/F 56 controls the exchange of various pieces ofinformation between the CPU 58 and the first smartphone 14A by thewireless communication method via the base station 20. In addition, thesecond communication I/F 56 controls the exchange of various pieces ofinformation between the CPU 58 and the first HMD 34A by the wirelesscommunication method via the base station 20. In addition, the secondcommunication I/F 56 controls the exchange of various pieces ofinformation between the CPU 58 and the second smartphone 14B by thewireless communication method via the base station 20. Further, thesecond communication I/F 56 controls the exchange of various pieces ofinformation between the CPU 58 and the second HMD 34B by the wirelesscommunication method via the base station 20.

For example, as shown in FIG. 5, the first smartphone 14A comprises acomputer 70, a GPS receiver 72, a gyro sensor 74, a reception device 76,a display 78, a microphone 80, a speaker 82, an imaging apparatus 84,and a communication I/F 86. The computer 70 comprises a CPU 88, astorage 90, and a memory 92, and the CPU 88, the storage 90, and thememory 92 are connected to each other via a bus line 94. In the exampleshown in FIG. 5, for convenience of illustration, one bus line is shownas the bus line 94, but a data bus, an address bus, a control bus, andthe like are included in the bus line 94.

The CPU 88 controls the whole first smartphone 14A. Various parametersand various programs are stored in the storage 90. The storage 90 is anon-volatile storage device. Here, an EEPROM is adopted as an example ofthe storage 90, but the technology of the present disclosure is notlimited to this, and a mask ROM, an HDD, an SSD, or the like may beused. The memory 92 is a volatile storage device. Various pieces ofinformation are transitorily stored in the memory 92, and the memory 92is used as a work memory by the CPU 88. Here, a DRAM is adopted as anexample of the memory 92, but the technology of the present disclosureis not limited to this, and another type of volatile storage device,such as an SRAM, may be used.

The GPS receiver 72 receives radio waves from a plurality of GPSsatellites (not shown) in response to the instruction from the CPU 88,and outputs reception result information indicating a reception resultto the CPU 88. The CPU 88 calculates current position informationindicating a current position of the first smartphone 14A as thethree-dimensional coordinate based on the reception result informationinput from the GPS receiver 72.

The gyro sensor 74 measures an angle around a yaw axis of the firstsmartphone 14A (hereinafter, also referred to as “yaw angle”), an anglearound a roll axis of the first smartphone 14A (hereinafter, alsoreferred to as “roll angle”), and an angle around a pitch axis of thefirst smartphone 14A (hereinafter, also referred to as “pitch angle”).The gyro sensor 74 is connected to the bus line 94, and angleinformation indicating the yaw angle, the roll angle, and the pitchangle measured by the gyro sensor 74 is acquired by the CPU 88 via thebus line 94. Note that the first smartphone 14A also comprises anacceleration sensor (not shown). Note that the acceleration sensor andthe gyro sensor 74 may be installed as an integrated multi-axes (forexample, 6 axes) sensor.

The reception device 76 receives the instruction from the viewer 28A.Examples of the reception device 76 include a touch panel 76A, and ahard key. The reception device 76 is connected to the bus line 94, andthe CPU 88 acquires the instruction received by the reception device 76.

The display 78 is connected to the bus line 94 and displays variouspieces of information under the control of the CPU 88. Examples of thedisplay 78 include a liquid crystal display. Note that another type ofdisplay, such as an organic EL display, may be adopted as the display 78without being limited to the liquid crystal display.

The first smartphone 14A comprises a touch panel display, and the touchpanel display is realized by the touch panel 76A and the display 78.That is, the touch panel display is formed by superimposing the touchpanel 76A on a display region of the display 78.

The microphone 80 converts a collected sound into an electric signal.The microphone 80 is connected to the bus line 94. The CPU 88 acquiresthe electric signal obtained by converting the sound collected by themicrophone 80 via the bus line 94.

The speaker 82 converts the electric signal into the sound. The speaker82 is connected to the bus line 94. The speaker 82 receives the electricsignal output from the CPU 88 via the bus line 94, converts the receivedelectric signal into the sound, and outputs the sound obtained byconverting the electric signal to the outside of the first smartphone14A.

The imaging apparatus 84 acquires an image showing a subject by imagingthe subject. The imaging apparatus 84 is connected to the bus line 94.The image obtained by imaging the subject by the imaging apparatus 84 isacquired by the CPU 88 via the bus line 94.

The communication I/F 86 is connected to the base station 20 in thewirelessly communicable manner. The communication I/F 86 is realized bya device having an FPGA, for example. The communication I/F 86 isconnected to the bus line 94. The communication I/F 86 controls theexchange of various pieces of information between the CPU 88 and anexternal device by the wireless communication method via the basestation 20. Here, examples of the “external device” include the displaycontrol device 12, the unmanned aerial vehicle 27, the second smartphone14B, the first HMD 34A, and the second HMD 34B.

The second smartphone 14B has the same configuration as the firstsmartphone 14A. That is, the second smartphone 14B comprises a computer100, a GPS receiver 102, a gyro sensor 104, a reception device 106, atouch panel 106A, a display 108, a microphone 110, a speaker 112, animaging apparatus 114, a communication I/F 116, a CPU 118, a storage120, a memory 122, and a bus line 124.

The computer 100 corresponds to the computer 70. The GPS receiver 102corresponds to the GPS receiver 72. The gyro sensor 104 corresponds tothe gyro sensor 74. The reception device 106 corresponds to thereception device 76. The touch panel 106A corresponds to the touch panel76A. The display 108 corresponds to the display 78. The microphone 110corresponds to the microphone 80. The speaker 112 corresponds to thespeaker 82. The imaging apparatus 114 corresponds to the imagingapparatus 84. The communication I/F 116 corresponds to the communicationI/F 86. The CPU 118 corresponds to the CPU 88. The storage 120corresponds to the storage 90. The memory 122 corresponds to the memory92. The bus line 124 corresponds to the bus line 94. Similar to the buslines 64 and 94, the bus line 124 also includes a data bus, an addressbus, a control bus, and the like.

For example, as shown in FIG. 6, the first HMD 34A comprises thecomputer 150, a reception device 152, a display 154, a microphone 157, aspeaker 158, an eye tracker 166, and a communication I/F 168. Thecomputer 150 comprises a CPU 160, a storage 162, and a memory 164, andthe CPU 160, the storage 162, and the memory 164 are connected via a busline 170. In the example shown in FIG. 6, for convenience ofillustration, one bus line is shown as the bus line 170, but a data bus,an address bus, a control bus, and the like are included in the bus line170.

The CPU 160 controls the whole first HMD 34A. Various parameters andvarious programs are stored in the storage 162. The storage 162 is anon-volatile storage device. Here, an EEPROM is adopted as an example ofthe storage 162, but the technology of the present disclosure is notlimited to this, and a mask ROM, an HDD, an SSD, or the like may beused. The memory 164 is a volatile storage device. Various pieces ofinformation are transitorily stored in the memory 164, and the memory164 is used as a work memory by the CPU 160. Here, a DRAM is adopted asan example of the memory 164, but the technology of the presentdisclosure is not limited to this, and another type of volatile storagedevice, such as an SRAM, may be used.

The reception device 152 receives the instruction from the viewer 28A.Examples of the reception device 152 include a remote controller and/ora hard key. The reception device 152 is connected to the bus line 170,and the CPU 160 acquires the instruction received by the receptiondevice 152.

The display 154 is a display capable of displaying the distributionvideo visually recognized by the viewer 28A, and is a display capable ofdisplaying a first viewpoint video selected from among a plurality ofviewpoint videos 46 (see FIG. 8), which will be described below. Thedisplay 154 is connected to the bus line 170 and displays various piecesof information under the control of the CPU 160. Examples of the display154 include a liquid crystal display. Note that another type of display,such as an organic EL display, may be adopted as the display 154 withoutbeing limited to the liquid crystal display. Note that the display 154is an example of a “first display unit (first display)” according to thetechnology of the present disclosure.

The eye tracker 166 includes an imaging apparatus (not shown), imagesboth eyes of the viewer 28A depending on a predetermined frame rate (forexample, 60 fps) by using the imaging apparatus, and detects theviewpoint position and the visual line direction of the viewer 28A basedon the image obtained by imaging. Then, the eye tracker 166 specifies agazing point at which the viewer 28A gazes in the distribution videodisplayed on the display 154 based on the detected viewpoint positionand visual line direction.

The communication I/F 168 is connected to the base station 20 in awirelessly communicable manner. The communication I/F 168 is realized bya device having an FPGA, for example. The communication I/F 168 isconnected to the bus line 170. The communication I/F 168 controls theexchange of various pieces of information between the CPU 160 and anexternal device by the wireless communication method via the basestation 20. Here, examples of the “external device” include the displaycontrol device 12, the unmanned aerial vehicle 27, the first smartphone14A, the second smartphone 14B, and the second HMD 34B.

The second HMD 34B has the same configuration as the first HMD 34A. Thatis, the second HMD 34B comprises the computer 200, a reception device202, a display 204, a microphone 207, a speaker 208, a CPU 210, astorage 212, a memory 214, an eye tracker 216, a communication I/F 218,and a bus line 220.

The computer 200 corresponds to the computer 150. The reception device202 corresponds to the reception device 152. The display 204 correspondsto the display 154. The microphone 207 corresponds to the microphone157. The speaker 208 corresponds to the speaker 158. The CPU 210corresponds to the CPU 160. The storage 212 corresponds to the storage162. The memory 214 corresponds to the memory 164. The eye tracker 216corresponds to the eye tracker 166. The communication I/F 218corresponds to the communication I/F 168. The bus line 220 correspondsto the bus line 170. Similar to the bus lines 64, 94 and 170, the busline 220 includes a data bus, an address bus, a control bus, and thelike.

The display 204 is a display capable of displaying the distributionvideo visually recognized by the viewer 28B, and is a display capable ofdisplaying a second viewpoint video selected from among the plurality ofviewpoint videos 46 (see FIG. 8), which will be described below. Notethat the display 204 is an example of a “second display unit (seconddisplay)” according to the technology of the present disclosure.

For example, as shown in FIG. 7, in the display control device 12, thestorage 60 stores a first display control program 60A, a second displaycontrol program 60B, and a setting program 60C. Note that, in thefollowing, in a case in which a distinction is not necessary, the firstdisplay control program 60A, the second display control program 60B, andthe setting program 60C are referred to as “display control deviceprogram” without reference numeral.

The CPU 58 reads out the display control device program from the storage60, and expands the readout display control device program in the memory62. The CPU 58 controls the whole display control device 12 according tothe display control device program expanded in the memory 62, andexchanges various pieces of information with the plurality of imagingapparatuses, the unmanned aerial vehicle 27, and the terminal device.

The CPU 58 is an example of a “processor” according to the technology ofthe present disclosure, and the memory 62 is an example of a “memory”according to the technology of the present disclosure. The CPU 58 readsout the first display control program 60A from the storage 60, andexpands the readout first display control program 60A in the memory 62.The CPU 58 is operated as a first acquisition unit 58A and a firstcontrol unit 58B according to the first display control program 60Aexpanded in the memory 62. The CPU 58 is operated as the firstacquisition unit 58A and the first control unit 58B to execute a firstdisplay control process (see FIGS. 35 to 37), which will be describedbelow.

The CPU 58 reads out the second display control program 60B from thestorage 60, and expands the readout second display control program 60Bin the memory 62. The CPU 58 is operated as a second acquisition unit58C and a second control unit 58D according to the second displaycontrol program 60B expanded in the memory 62. The CPU 58 is operated asthe second acquisition unit 58C and the second control unit 58D toexecute a second display control process (see FIGS. 38 to 40), whichwill be described below.

The CPU 58 reads out the setting program 60C from the storage 60, andexpands the readout setting program 60C in the memory 62. The CPU 58 isoperated as a setting unit 58E according to the setting program 60Cexpanded in the memory 62. The CPU 58 is operated as the setting unit58E to execute a setting process (see FIG. 41), which will be describedbelow.

For example, as shown in FIG. 8, in the display control device 12, theCPU 58 executes a viewpoint video generation process. The viewpointvideo generation process is a process of generating the plurality ofviewpoint videos 46. The distribution video described above includes theplurality of viewpoint videos 46. Each of the plurality of viewpointvideos 46 is a video showing the imaging region observed from thecorresponding peculiar viewpoint. The plurality of viewpoint videos 46include the bird's-eye view video 46A, the imaging video 46B, and thevirtual viewpoint video 46C. The virtual viewpoint video 46C isgenerated based on the bird's-eye view video 46A acquired by the imagingapparatus 18 and the plurality of imaging videos 46B acquired by theplurality of imaging apparatuses 16.

Note that, here, although an aspect example is described in which thevirtual viewpoint video 46C is generated based on the bird's-eye viewvideo 46A and the plurality of imaging videos 46B, the technology of thepresent disclosure is not limited to this, and the virtual viewpointvideo 46C may be generated based on at least two or more videos of thebird's-eye view video 46A and the plurality of imaging videos 46B. Inaddition, here, an aspect example is described in which the bird's-eyeview video 46A, the imaging video 46B, and the virtual viewpoint video46C are included in the plurality of viewpoint videos 46, but thetechnology of the present disclosure is not limited to this, it is notnecessary for the virtual viewpoint video 46C to be included in theplurality of viewpoint videos 46, and it is not necessary for thebird's-eye view video 46A to be included in the plurality of viewpointvideos 46. In addition, it is not necessary for the CPU 58 to acquirethe imaging videos 46B from all of the plurality of imaging apparatuses16, and it is not necessary to acquire a part of the imaging videos 46B.

In addition, the viewpoint video 46 is displayed on each display, suchas the display 78 (see FIG. 5) of the first smartphone 14A, the display108 of the second smartphone 14B, the display 154 of the first HMD 34A,and the display 204 of the second HMD 34B. A size of the viewpoint video46 generated by the viewpoint video generation process and a displaysize of the viewpoint video 46 displayed on each display have asimilarity relationship. In addition, in the following description, achange of the size of the viewpoint video 46 refers to a change of thedisplay size of the viewpoint video 46. Note that the same applies to anavatar, which will be described below, and a change of a size of theavatar refers to a change of a display size of the avatar.

For example, as shown in FIG. 9, each of the plurality of viewpointvideos 46 obtained by executing the viewpoint video generation processby the CPU 58 has a viewpoint video identifier, a peculiar viewpointposition information, peculiar visual line direction information, andpeculiar angle-of-view information. That is, each of the plurality ofviewpoint videos 46 is associated with the viewpoint video identifier,the peculiar viewpoint position information, and the peculiar visualline direction information. The viewpoint video identifier is anidentifier that can uniquely specify the corresponding viewpoint video46. The peculiar viewpoint position information is informationindicating the peculiar viewpoint position. The peculiar viewpointposition is the viewpoint position of the corresponding viewpoint video46. That is, the peculiar viewpoint position refers to the viewpointposition during which the imaging region shown by the correspondingviewpoint video 46 is observed. Examples of the peculiar viewpointposition information include the three-dimensional coordinate in whichthe peculiar viewpoint positions of the plurality of viewpoint videos 46can be relatively specified. Note that, here, the peculiar viewpointposition is limited to a partial region of the imaging region. Thepartial region refers to the spectator seat 26 (see FIGS. 1 and 3), forexample.

The peculiar visual line direction information is information indicatinga peculiar visual line direction. The peculiar visual line direction isthe visual line direction of the corresponding viewpoint video 46. Thevisual line direction included in the corresponding viewpoint video 46refers to the visual line direction in which the imaging region shown bythe corresponding viewpoint video 46 is observed. Here, as an example ofthe peculiar visual line direction, a direction facing the correspondingviewpoint video 46 (for example, a direction passing through a center ofthe viewpoint video 46 perpendicularly to the viewpoint video 46) isadopted.

The peculiar angle-of-view information is information indicating apeculiar angle of view. The peculiar angle of view is the angle of viewof the corresponding viewpoint video 46. That is, the peculiar angle ofview refers to the angle of view with respect to the imaging regionshown by the corresponding viewpoint video 46.

For example, as shown in FIG. 10, in the display control device 12, theCPU 58 executes a bird's-eye view video transmission process. Thebird's-eye view video transmission process is a process of transmittingthe bird's-eye view video 46A among the plurality of viewpoint videos 46generated by the viewpoint video generation process to the firstsmartphone 14A and the second smartphone 14B.

The bird's-eye view video 46A is received by the first smartphone 14A,and the received bird's-eye view video 46A is displayed on the display78 of the first smartphone 14A. In a state in which the bird's-eye viewvideo 46A is displayed on the display 78, the viewer 28A gives a firstviewpoint visual line instruction to the first smartphone 14A. The touchpanel 76A of the first smartphone 14A is a device capable of receivingthe first viewpoint visual line instruction, and is an example of a“first reception unit (first reception device)” according to thetechnology of the present disclosure. The first viewpoint visual lineinstruction is an instruction for the viewpoint position and the visualline direction with respect to the imaging region, and is used as aninstruction for selecting any one of the plurality of viewpoint videos46. Examples of the first viewpoint visual line instruction include atouch operation and a slide operation on the touch panel 76A. In thiscase, the instruction for the visual line direction is given byperforming the touch operation on the touch panel 76A, and theinstruction for the viewpoint position is given by performing the slideoperation on the touch panel 76A. The position at which the touchoperation is performed on the touch panel 76A corresponds to theviewpoint position with respect to the imaging region, and the directionin which the slide operation is performed on the touch panel 76Acorresponds to the visual line direction with respect to the imagingregion. Note that the first viewpoint visual line instruction is anexample of a “first instruction” according to the technology of thepresent disclosure.

The bird's-eye view video 46A is received by the second smartphone 14B,and the received bird's-eye view video 46A is displayed on the display108 of the second smartphone 14B. In a state in which the bird's-eyeview video 46A is displayed on the display 108, the viewer 28B gives asecond viewpoint visual line instruction to the second smartphone 14B.The touch panel 106A of the second smartphone 14B is a device capable ofreceiving the second viewpoint visual line instruction, and is anexample of a “second reception unit (second reception device)” accordingto the technology of the present disclosure. The second viewpoint visualline instruction is an instruction for the viewpoint position and thevisual line direction with respect to the imaging region, and is used asan instruction for selecting any one of the plurality of viewpointvideos 46. Examples of the second viewpoint visual line instructioninclude a touch operation and a slide operation on the touch panel 106A.In this case, the instruction for the visual line direction is given byperforming the touch operation on the touch panel 106A, and theinstruction for the viewpoint position is given by performing the slideoperation on the touch panel 106A. The position at which the touchoperation is performed on the touch panel 106A corresponds to theviewpoint position with respect to the imaging region, and the directionin which the slide operation is performed on the touch panel 106Acorresponds to the visual line direction with respect to the imagingregion. Note that the second viewpoint visual line instruction is anexample of a “second instruction” according to the technology of thepresent disclosure.

As shown in FIG. 11, for example, the first viewpoint visual lineinstruction received by the touch panel 76A is transmitted by the CPU 88to the CPU 58 of the display control device 12, and is received by theCPU 58. The second viewpoint visual line instruction received by thetouch panel 106A is transmitted by the CPU 118 to the CPU 58 of thedisplay control device 12, and is received by the CPU 58.

For example, as shown in FIG. 12, in the display control device 12, thefirst control unit 58B comprises a first viewpoint video acquisitionunit 58B1, a first determination unit 58B2, a first composing unit 58B3,a first avatar display size changing unit 58B4, a first image qualitycontrol unit 58B5, a first display aspect changing unit 58B6, and afirst viewpoint video output unit 58B7. The first control unit 58Bperforms a control of displaying the first viewpoint video selected fromamong the plurality of viewpoint videos 46 on the display 154 of thefirst HMD 34A. Here, the “first viewpoint video” refers to one viewpointvideo selected from among the plurality of viewpoint videos 46 by thefirst control unit 58B. According to the first display control program60A expanded in the memory 62, the CPU 58 is operated as the firstviewpoint video acquisition unit 58B1, the first determination unit58B2, the first composing unit 58B3, the first avatar display sizechanging unit 58B4, the first image quality control unit 58B5, the firstdisplay aspect changing unit 58B6, and the first viewpoint video outputunit 58B7.

In the display control device 12, the second control unit 58D comprisesa second viewpoint video acquisition unit 58D1, a second determinationunit 58D2, a second composing unit 58D3, a second avatar display sizechanging unit 58D4, a second image quality control unit 58D5, a seconddisplay aspect changing unit 58D6, and a second viewpoint video outputunit 58D7. The second control unit 58D performs a control of displayingthe first viewpoint video selected from among the plurality of viewpointvideos 46 on the display 204 of the second HMD 34B. Here, the “secondviewpoint video” refers to one viewpoint video selected from among theplurality of viewpoint videos 46 by the second control unit 58D.According to the second display control program 60B expanded in thememory 62, the CPU 58 is operated as the second viewpoint videoacquisition unit 58D1, the second determination unit 58D2, the secondcomposing unit 58D3, the second avatar display size changing unit 58D4,the second image quality control unit 58D5, the second display aspectchanging unit 58D6, and the second viewpoint video output unit 58D7.

For example, as shown in FIG. 13, the first viewpoint video acquisitionunit 58B1 receives the first viewpoint visual line instructiontransmitted from the first smartphone 14A. In a case in which the firstviewpoint visual line instruction is received, the first viewpoint videoacquisition unit 58B1 selects one viewpoint video 46 from among theplurality of viewpoint videos 46 as the first viewpoint video inresponse to the first viewpoint visual line instruction, and acquiresthe selected first viewpoint video. The first viewpoint video is theviewpoint video 46 having peculiar viewpoint position information andpeculiar visual line direction information corresponding to theviewpoint position and the visual line direction as instructed by thefirst viewpoint visual line instruction.

The first viewpoint video acquisition unit 58B1 acquires the viewpointvideo identifier associated with the acquired first viewpoint video andoutputs the acquired viewpoint video identifier to the first acquisitionunit 58A. The first acquisition unit 58A acquires the peculiar viewpointposition information, the peculiar visual line direction information,and the peculiar angle-of-view information associated with the viewpointvideo 46 specified by the viewpoint video identifier input from thefirst viewpoint video acquisition unit 58B1.

The memory 62 has a first storage region 62A and a second storage region62B. The first acquisition unit 58A stores the acquired peculiarviewpoint position information as the second viewpoint positioninformation in the first storage region 62A. The second viewpointposition information is the peculiar viewpoint position information ofany one of the plurality of viewpoint videos 46. Here, the secondviewpoint position information refers to information indicating a secondviewpoint position. The second viewpoint position refers to theviewpoint position of the viewer 28A with respect to the imaging region.

The first acquisition unit 58A stores the acquired peculiar visual linedirection information in the first storage region 62A as second visualline direction information. The second visual line direction informationis the peculiar visual line direction information of any one of theplurality of viewpoint videos 46. Here, the second visual line directioninformation refers to information indicating a second visual linedirection. The second visual line direction refers to a visual linedirection of the viewer 28A with respect to the imaging region.

The first acquisition unit 58A stores the acquired peculiarangle-of-view information in the first storage region 62A. In addition,the first acquisition unit 58A stores the viewpoint video identifierinput from the first viewpoint video acquisition unit 58B1 in the firststorage region 62A.

Note that, in a case in which the second viewpoint position information,the second visual line direction information, and the peculiarangle-of-view information are newly acquired by the first acquisitionunit 58A, the new second viewpoint position information, second visualline direction information, and peculiar angle-of-view information areoverwritten and stored on the first storage region 62A by the firstacquisition unit 58A. In a case in which the viewpoint video identifieris newly input to the first acquisition unit 58A from the firstviewpoint video acquisition unit 58B1, the new viewpoint videoidentifier is overwritten and stored on the first storage region 62A bythe first acquisition unit 58A.

The second viewpoint video acquisition unit 58D1 receives the secondviewpoint visual line instruction transmitted from the second smartphone14B. In a case in which the second viewpoint visual line instruction isreceived, the second viewpoint video acquisition unit 58D1 selects oneviewpoint video 46 from among the plurality of viewpoint videos 46 asthe second viewpoint video in response to the second viewpoint visualline instruction, and acquires the selected second viewpoint video. Thesecond viewpoint video is the viewpoint video 46 having peculiarviewpoint position information and peculiar visual line directioninformation corresponding to the viewpoint position and the visual linedirection as instructed by the second viewpoint visual line instruction.

The second viewpoint video acquisition unit 58D1 acquires the viewpointvideo identifier associated with the acquired second viewpoint video andoutputs the acquired viewpoint video identifier to the secondacquisition unit 58C. The second acquisition unit 58C acquires thepeculiar viewpoint position information, the peculiar visual linedirection information, and the peculiar angle-of-view informationassociated with the viewpoint video 46 specified by the viewpoint videoidentifier input from the second viewpoint video acquisition unit 58D1.

The second acquisition unit 58C stores the acquired peculiar viewpointposition information as the first viewpoint position information in thesecond storage region 62B. The first viewpoint position information isthe peculiar viewpoint position information of any one of the pluralityof viewpoint videos 46. Here, the first viewpoint position informationrefers to information indicating a first viewpoint position. The firstviewpoint position refers to the viewpoint position of the viewer 28Bwith respect to the imaging region.

The second acquisition unit 58C stores the acquired peculiar visual linedirection information in the second storage region 62B as first visualline direction information. The first visual line direction informationis the peculiar visual line direction information of any one of theplurality of viewpoint videos 46. Here, the first visual line directioninformation refers to information indicating a first visual linedirection. The first visual line direction refers to a visual linedirection of the viewer 28B with respect to the imaging region.

The second acquisition unit 58C stores the acquired peculiarangle-of-view information in the second storage region 62B. In addition,the second acquisition unit 58C stores the viewpoint video identifierinput from the second viewpoint video acquisition unit 58D1 in thesecond storage region 62B.

Note that, in a case in which the first viewpoint position information,the first visual line direction information, and the peculiarangle-of-view information are newly acquired by the second acquisitionunit 58C, the new first viewpoint position information, first visualline direction information, and peculiar angle-of-view information areoverwritten and stored on the second storage region 62B by the secondacquisition unit 58C. In a case in which the viewpoint video identifieris newly input to the second acquisition unit 58C from the secondviewpoint video acquisition unit 58D1, the new viewpoint videoidentifier is overwritten and stored on the second storage region 62B bythe second acquisition unit 58C.

For example, as shown in FIG. 14, the first acquisition unit 58Aacquires the first viewpoint position information. Specifically, thepeculiar viewpoint position information corresponding to the secondviewpoint video displayed on the display 204 of the second HMD 34B isstored in the second storage region 62B as the first viewpoint positioninformation, and the first viewpoint position information is acquiredfrom the second storage region 62B by the first acquisition unit 58A.

In a case in which the first viewpoint position indicated by the firstviewpoint position information acquired by the first acquisition unit58A is included in the first viewpoint video, the first control unit 58Bperforms a control of displaying a first person avatar (see FIG. 25) forspecifying the first viewpoint position indicated by the first viewpointposition information acquired by the first acquisition unit 58A in thefirst viewpoint video. In addition, the first control unit 58B performsa control of changing a display size of a first avatar depending on anangle of view of a first avatar-containing viewpoint video (see FIG. 31)displayed by the display 154.

In addition, the first control unit 58B performs a control of causingthe display 154 to set a degree of difference between an image qualityof the first viewpoint video and an image quality of the first avatarwithin a first predetermined range (see FIGS. 24 and 25). In addition,the first control unit 58B performs a control of causing the display 154to change a display aspect of the first avatar depending on arelationship between the display size of the first viewpoint video andthe display size of the first avatar. In addition, the first controlunit 58B performs a control of causing the display 154 to change thedisplay aspect of the first avatar in a case in which a ratio of thedisplay size of the first avatar to the display size of the firstviewpoint video is equal to or more than a first predetermined value(for example, 5 percent) (see FIGS. 27 to 29). Note that the firstpredetermined value is an example of a “first threshold value” of thetechnology of the present disclosure. Hereinafter, a specificdescription will be made.

The first viewpoint video acquisition unit 58B1 outputs the acquiredfirst viewpoint video to the first determination unit 58B2. In a case inwhich the first viewpoint video is input from the first viewpoint videoacquisition unit 58B1, the first determination unit 58B2 requests thefirst acquisition unit 58A to acquire the first viewpoint positioninformation. The first acquisition unit 58A acquires the first viewpointposition information from the second storage region 62B in response tothe request from the first determination unit 58B2 and outputs theacquired first viewpoint position information to the first determinationunit 58B2.

The first determination unit 58B2 determines whether or not the firstviewpoint position indicated by the first viewpoint position informationinput from the first acquisition unit 58A is included in the firstviewpoint video. The determination as to whether or not the firstviewpoint position is included in the first viewpoint video is made byreferring to, for example, the peculiar angle-of-view information, thepeculiar viewpoint position information, and the peculiar visual linedirection information corresponding to the first viewpoint video.

Here, the fact that the first viewpoint position is included in thefirst viewpoint video refers to, for example, that at the current pointin time, the viewpoint position of the viewer 28B who visuallyrecognizes the second viewpoint video is included in the first viewpointvideo visually recognized by the viewer 28A, as shown in FIG. 15. Notethat the viewers 28A and 28B shown in FIG. 15 are virtual images shownfor convenience, and in the example shown in FIG. 15, it merelyexpresses virtual presence positions of the viewers 28A and 28B, whichcan be specified, with respect to the soccer field 24.

For example, as shown in FIG. 16, the second acquisition unit 58Cacquires the second viewpoint position information. Specifically, thepeculiar viewpoint position information corresponding to the firstviewpoint video displayed on the display 154 of the first HMD 34A isstored in the first storage region 62A as the second viewpoint positioninformation, and the second viewpoint position information is acquiredfrom the first storage region 62A by the second acquisition unit 58C.

In a case in which the second viewpoint position indicated by the secondviewpoint position information acquired by the second acquisition unit58C is included in the second viewpoint video, the second control unit58D performs a control of displaying a second person avatar forspecifying the second viewpoint position indicated by the secondviewpoint position information acquired by the second acquisition unit58C in the second viewpoint video. In addition, the second control unit58D performs a control of changing a display size of a second avatardepending on an angle of view of a second avatar-containing viewpointvideo (see FIG. 32) displayed by the display 204.

In addition, the second control unit 58D performs a control of causingthe display 204 to set a degree of difference between an image qualityof the second viewpoint video and an image quality of the second avatarwithin a second predetermined range. In addition, the second controlunit 58D performs a control of causing the display 204 to change adisplay aspect of the second avatar depending on a relationship betweenthe display size of the second viewpoint video and the display size ofthe second avatar. In addition, the second control unit 58D performs acontrol of causing the display 204 to change the display aspect of thesecond avatar in a case in which a ratio of the display size of thesecond avatar to the display size of the second viewpoint video is equalto or more than a second predetermined value (for example, 5 percent)(see FIGS. 29 and 30). Note that the second predetermined value is anexample of a “third threshold value” of the technology of the presentdisclosure. Hereinafter, a specific description will be made.

The second viewpoint video acquisition unit 58D1 outputs the acquiredsecond viewpoint video to the second determination unit 58D2. In a casein which the second viewpoint video is input from the second viewpointvideo acquisition unit 58D1, the second determination unit 58D2 requeststhe second acquisition unit 58C to acquire the second viewpoint positioninformation. The second acquisition unit 58C acquires the secondviewpoint position information from the first storage region 62A inresponse to the request from the second determination unit 58D2 andoutputs the acquired second viewpoint position information to the seconddetermination unit 58D2.

The second determination unit 58D2 determines whether or not the secondviewpoint position indicated by the second viewpoint positioninformation input from the second acquisition unit 58C is included inthe second viewpoint video. The determination as to whether or not thesecond viewpoint position is included in the second viewpoint video ismade by referring to, for example, the peculiar angle-of-viewinformation, the peculiar viewpoint position information, and thepeculiar visual line direction information corresponding to the secondviewpoint video.

Here, the fact that the second viewpoint position is included in thesecond viewpoint video refers to, for example, that at the current pointin time, the viewpoint position of the viewer 28A who visuallyrecognizes the first viewpoint video is included in the second viewpointvideo visually recognized by the viewer 28B, as shown in FIG. 17. Notethat the viewers 28A and 28B shown in FIG. 17 are virtual images shownfor convenience, and in the example shown in FIG. 17, it merelyexpresses virtual presence positions of the viewers 28A and 28B, whichcan be specified, with respect to the soccer field 24.

For example, as shown in FIG. 18, in a case in which it is determinedthat the first viewpoint position is included in the first viewpointvideo, the first determination unit 58B2 outputs first person presenceinformation indicating that the viewer 28B is present in a field of viewof the viewer 28A, that is, the first viewpoint position is present tothe first composing unit 58B3. In a case in which it is determined thatthe first viewpoint position is not included in the first viewpointvideo, the first determination unit 58B2 outputs first person absenceinformation indicating that the viewer 28B is not present in a field ofview of the viewer 28A, that is, the first viewpoint position is notpresent to the first viewpoint video acquisition unit 58B 1.

In a case in which the first person absence information is input fromthe first determination unit 58B2, the first viewpoint video acquisitionunit 58B1 outputs the first viewpoint video to the first viewpoint videooutput unit 58B7. The first viewpoint video output unit 58B7 outputs thefirst viewpoint video input from the first viewpoint video acquisitionunit 58B1 to the first HMD 34A. The first viewpoint video output unit58B7 outputs the first viewpoint video to the first HMD 34A, and thusthe first viewpoint video is displayed on the display 154.

A first reference avatar group is stored in the storage 60. The firstreference avatar group is a set of a plurality of first referenceavatars. The first reference avatar refers to a virtual image inimitation of the viewer 28B. The first reference avatar group includesthe plurality of first reference avatars indicating the viewer 28B in acase in which the viewer 28B is observed from a plurality of directions.

In a case in which the first person presence information is input fromthe first determination unit 58B2, the first composing unit 58B3requests the first acquisition unit 58A to acquire the first viewpointposition information and the first visual line direction information.The first acquisition unit 58A acquires the first viewpoint positioninformation and the first visual line direction information from thesecond storage region 62B in response to the request from the firstcomposing unit 58B3 and outputs the acquired first viewpoint positioninformation and first visual line direction information to the firstcomposing unit 58B3.

In a case in which the first person presence information is input fromthe first determination unit 58B2, the first composing unit 58B3acquires the first viewpoint video from the first viewpoint videoacquisition unit 58B1 and acquires the first reference avatar group fromthe storage 60.

The first composing unit 58B3 generates the first avatar based on thefirst visual line direction information input from the first acquisitionunit 58A. The first avatar refers to a virtual image in imitation of theviewer 28B. By using the first reference avatar group, the firstcomposing unit 58B3 generates the first avatar for specifying the firstvisual line direction indicated by the first visual line directioninformation. That is, the first composing unit 58B3 composes the firstreference avatar group to generate the first avatar indicating theviewer 28B who faces the first visual line direction. In this way, thefirst avatar is generated by the first composing unit 58B3 as the avatarfor specifying the first visual line direction indicated by the firstvisual line direction information acquired by the first acquisition unit58A. Note that the first avatar is an example of “first specificinformation” according to the technology of the present disclosure.

The first composing unit 58B3 generates the first avatar-containingviewpoint video by superimposing the first avatar on the first viewpointvideo at the first viewpoint position indicated by the first viewpointposition information input from the first acquisition unit 58A, forexample, as shown in FIG. 19. The first composing unit 58B3 outputs thegenerated first avatar-containing viewpoint video to the first avatardisplay size changing unit 58B4. Note that, here, although an aspectexample is described in which the first avatar is superimposed on thefirst viewpoint video, the technology of the present disclosure is notlimited to this. For example, the first avatar may be embedded in thefirst viewpoint video, finally, the first avatar need only be displayedin the second viewpoint video displayed on the display 154 of the firstHMD 34A.

For example, as shown in FIG. 20, in a case in which it is determinedthat the second viewpoint position is included in the second viewpointvideo, the second determination unit 58D2 outputs second person presenceinformation indicating that the viewer 28A is present in a field of viewof the viewer 28B, that is, the second viewpoint position is present tothe second composing unit 58D3. In a case in which it is determined thatthe second viewpoint position is not included in the second viewpointvideo, the second determination unit 58D2 outputs second person absenceinformation indicating that the viewer 28A is not present in a field ofview of the viewer 28B, that is, the second viewpoint position is notpresent to the second viewpoint video acquisition unit 58D1.

In a case in which the second person absence information is input fromthe second determination unit 58D2, the second viewpoint videoacquisition unit 58D1 outputs the second viewpoint video to the secondviewpoint video output unit 58D7. The second viewpoint video output unit58D7 outputs the second viewpoint video input from the second viewpointvideo acquisition unit 58D1 to the second HMD 34B. The second viewpointvideo output unit 58D7 outputs the second viewpoint video to the secondHMD 34B, and thus the second viewpoint video is displayed on the display204.

A second reference avatar group is stored in the storage 60. The secondreference avatar group is a set of a plurality of second referenceavatars. The second reference avatar refers to a virtual image inimitation of the viewer 28A. The second reference avatar group includesthe plurality of second reference avatars indicating the viewer 28A in acase in which the viewer 28A is observed from a plurality of directions.

In a case in which the second person presence information is input fromthe second determination unit 58D2, the second composing unit 58D3requests the second acquisition unit 58C to acquire the second viewpointposition information and the second visual line direction information.The second acquisition unit 58C acquires the second viewpoint positioninformation and the second visual line direction information from thefirst storage region 62A in response to the request from the secondcomposing unit 58D3 and outputs the acquired second viewpoint positioninformation and second visual line direction information to the secondcomposing unit 58D3.

In a case in which the second person presence information is input fromthe second determination unit 58D2, the second composing unit 58D3acquires the second viewpoint video from the second viewpoint videoacquisition unit 58D1 and acquires the second reference avatar groupfrom the storage 60.

The second composing unit 58D3 generates the second avatar based on thesecond visual line direction information input from the secondacquisition unit 58C. The second avatar refers to a virtual image inimitation of the viewer 28A. By using the second reference avatar group,the second composing unit 58D3 generates the second avatar forspecifying the second visual line direction indicated by the secondvisual line direction information. That is, the second composing unit58D3 composes the second reference avatar group to generate the secondavatar indicating the viewer 28A who faces the second visual linedirection. In this way, the second avatar is generated by the secondcomposing unit 58D3 as the avatar for specifying the second visual linedirection indicated by the second visual line direction informationacquired by the second acquisition unit 58C. Note that the second avataris an example of “second specific information” according to thetechnology of the present disclosure.

The second composing unit 58D3 generates the second avatar-containingviewpoint video by superimposing the second avatar on the secondviewpoint video at the second viewpoint position indicated by the secondviewpoint position information input from the second acquisition unit58C, as shown in FIG. 19. The second composing unit 58D3 outputs thegenerated second avatar-containing viewpoint video to the second avatardisplay size changing unit 58D4. Note that, here, although an aspectexample is described in which the second avatar is superimposed on thesecond viewpoint video, the technology of the present disclosure is notlimited to this. For example, the second avatar may be embedded in thesecond viewpoint video, finally, the second avatar need only bedisplayed in the second viewpoint video displayed on the display 206 ofthe second HMD 34B. Note that, in the following, for convenience ofdescription, in a case in which a distinction is not necessary, thefirst avatar and the second avatar are simply referred to as the“avatar”.

For example, as shown in FIG. 21, in a case in which the firstavatar-containing viewpoint video is input from the first composing unit58B3, the first avatar display size changing unit 58B4 requests thefirst acquisition unit 58A to acquire the peculiar angle-of-viewinformation. The first acquisition unit 58A acquires the peculiarangle-of-view information from the second storage region 62B in responseto the request from the first avatar display size changing unit 58B4 andoutputs the acquired peculiar angle-of-view information as the firstavatar display size changing unit 58B4.

The storage 60 stores a size derivation table for the avatar. The sizederivation table for the avatar is a table in which the angle of view ofthe viewpoint video 46 is associated with the size of the avatar. Thesize of the avatar refers to an area of the avatar, for example. Arelationship between the angle of view of the viewpoint video 46 and thesize of the avatar in the size derivation table for the avatar can bechanged in response to the instruction received by the reception device152. Here, the size derivation table is described as an example, but thetechnology of the present disclosure is not limited to this, and it isalso possible to apply an arithmetic expression with the angle of viewof the viewpoint video 46 as an independent variable and the size of theavatar as a dependent variable.

The first avatar display size changing unit 58B4 changes the size of thefirst avatar in the first avatar-containing viewpoint video to the sizedepending on the angle of view indicated by the peculiar angle-of-viewinformation. In this case, the first avatar display size changing unit58B4 derives the first avatar size from the size derivation table forthe avatar depending on the peculiar angle-of-view information. As thesize of the first avatar, the size corresponding to the angle of viewindicated by the peculiar angle-of-view information is derived from thesize derivation table for the avatar. The first avatar display sizechanging unit 58B4 changes the first avatar in the firstavatar-containing viewpoint video to have the size derived from the sizederivation table for the avatar.

For example, the size of the first avatar in the first avatar-containingviewpoint video shown in FIG. 19 is changed to the size of the firstavatar in the first avatar-containing viewpoint video shown in FIG. 22by the first avatar display size changing unit 58B4. In the exampleshown in FIG. 19, the size of the first avatar is several times or morelarger than the size of the spectator in the first avatar-containingviewpoint video, whereas in the example shown in FIG. 22, the size ofthe first avatar is changed to almost same size as the spectator in thefirst avatar-containing viewpoint video. As a result, in a case in whichthe first avatar-containing viewpoint video is displayed on the display154 of the first HMD 34A, the display size of the first avatar ischanged depending on the angle of view of the first avatar-containingviewpoint video displayed on the display 154.

For example, as shown in FIG. 23, in a case in which the secondavatar-containing viewpoint video is input from the second composingunit 58D3, the second avatar display size changing unit 58D4 requeststhe second acquisition unit 58C to acquire the peculiar angle-of-viewinformation. The second acquisition unit 58C acquires the peculiarangle-of-view information from the first storage region 62A in responseto the request from the second avatar display size changing unit 58D4and outputs the acquired peculiar angle-of-view information as thesecond avatar display size changing unit 58D4.

The second avatar display size changing unit 58D4 changes the size ofthe second avatar in the second avatar-containing viewpoint video to thesize depending on the angle of view indicated by the peculiarangle-of-view information in the same manner as the method in which thesize of the first avatar in the first avatar-containing viewpoint videois changed. That is, the second avatar display size changing unit 58D4derives the second avatar size from the size derivation table for theavatar depending on the peculiar angle-of-view information. As the sizeof the second avatar, the size corresponding to the angle of viewindicated by the peculiar angle-of-view information is derived from thesize derivation table for the avatar. The second avatar display sizechanging unit 58D4 changes the second avatar in the secondavatar-containing viewpoint video to have the size derived from the sizederivation table for the avatar. As a result, in a case in which thesecond avatar-containing viewpoint video is displayed on the display 204of the second HMD 34B, the display size of the second avatar is changeddepending on the angle of view of the second avatar-containing viewpointvideo displayed on the display 204.

For example, as shown in FIG. 24, the first avatar display size changingunit 58B4 outputs the first avatar-containing viewpoint video obtainedby changing the size of the first avatar to the size depending on theangle of view to the first image quality control unit 58B5. The firstimage quality control unit 58B5 controls the image quality of the firstavatar-containing viewpoint video such that the degree of differencebetween the image quality of the first viewpoint video and the imagequality of the first avatar is within the first predetermined range inthe first avatar-containing viewpoint video. Here, the “degree ofdifference” may be a difference between the image quality of the firstviewpoint video and the image quality of the first avatar, or may be aratio one of the image quality of the first viewpoint video or the imagequality of the first avatar to the other thereof.

Specifically, the first image quality control unit 58B5 determineswhether or not the degree of difference between the image quality of thefirst viewpoint video and the image quality of the first avatar iswithin the first predetermined range in the first avatar-containingviewpoint video. Then, in a case in which the degree of differencebetween the image quality of the first viewpoint video and the imagequality of the first avatar is not within the first predetermined range,the first image quality control unit 58B5 controls the image quality ofthe first avatar-containing viewpoint video to set the degree ofdifference between the image quality of the first viewpoint video andthe image quality of the first avatar within the first predeterminedrange.

Here, the “image quality” refers to a resolution, a contrast, and alightness and darkness degree. In addition, the fact that the degree ofdifference between the image quality of the first viewpoint video andthe image quality of the first avatar is within the first predeterminedrange refers to the fact that a degree of difference of the resolutionsis within a predetermined resolution range, a degree of difference ofthe contrasts is within a predetermined contrast range, and a degree ofdifference of the lightness and darkness degrees is within apredetermined lightness and darkness degree range. The predeterminedresolution range, the predetermined contrast range, the predeterminedlightness and darkness degree range may be fixed values or variablevalues. Examples of the fixed value include a value derived in advanceby a sensory test and/or a computer simulation as a value that does notcause a visual discomfort in a case in which the avatar enters theviewpoint video 46. Examples of the variable value include a value thatcan be changed in response to the instruction received by the receptiondevice 52, 76, 106, 152, or 202.

In the first avatar-containing viewpoint video, by controlling the imagequality of the first avatar-containing viewpoint video such thatdifference between the image quality of the first viewpoint video andthe image quality of the first avatar are within the first predeterminedrange, for example, the image quality of the first avatar shown in FIG.22 is changed as shown in FIG. 25. That is, the image quality of thefirst avatar is changed such that a visual discomfort is not caused dueto the first avatar appearing from the first viewpoint video or sinkinginto the first viewpoint video.

For example, as shown in FIG. 26, the second avatar display sizechanging unit 58D4 outputs the second avatar-containing viewpoint videoobtained by changing the size of the second avatar to the size dependingon the angle of view to the second image quality control unit 58D5. Thesecond image quality control unit 58D5 controls the image quality of thesecond avatar-containing viewpoint video to set the degree of differencebetween the image quality of the second viewpoint video and the imagequality of the second avatar within the second predetermined range inthe second avatar-containing viewpoint video in the same manner as themethod by the first image quality control unit 58B5. Here, the “degreeof difference” may be a difference between the image quality of thesecond viewpoint video and the image quality of the second avatar, ormay be a ratio one of the image quality of the second viewpoint video orthe image quality of the second avatar to the other thereof.

Specifically, the second image quality control unit 58D5 determineswhether or not the degree of difference between the image quality of thesecond viewpoint video and the image quality of the second avatar iswithin the second predetermined range in the second avatar-containingviewpoint video. Then, in a case in which the degree of differencebetween the image quality of the second viewpoint video and the imagequality of the second avatar is not within the second predeterminedrange, the second image quality control unit 58D5 controls the imagequality of the second avatar-containing viewpoint video to set thedegree of difference between the image quality of the second viewpointvideo and the image quality of the second avatar within the secondpredetermined range.

Here, the fact that the degree of difference between the image qualityof the second viewpoint video and the image quality of the second avataris within the second predetermined range refers to the fact that thedegree of difference of the resolutions is within the predeterminedresolution range, the degree of difference of the contrasts is withinthe predetermined contrast range, and the degree of difference of thelightness and darkness degrees is within the predetermined lightness anddarkness degree range, in the same manner as the relationship betweenthe degree of difference between the image quality of the firstviewpoint video and image quality of the first avatar, and the firstpredetermined range.

In the second avatar-containing viewpoint video, by controlling theimage quality of the second avatar-containing viewpoint video such thatdifference between the image quality of the second viewpoint video andthe image quality of the second avatar are within the secondpredetermined range, the image quality of the second avatar is changedsuch that a visual discomfort is not caused due to the second avatarappearing from the second viewpoint video or sinking into the secondviewpoint video.

Note that, here, although the resolution, the contrast, and thelightness and darkness degree are described as an example of the “imagequality”, the technology of the present disclosure is not limited tothis, and at least one or two of the resolution, the contrast, or thelightness and darkness degree may be used. In addition, the imagequality may be a factor that affects the image quality other than theresolution, the contrast, and the lightness and darkness degree.

For example, as shown in FIG. 27, the first image quality control unit58B5 outputs the first avatar-containing viewpoint video to the firstdisplay aspect changing unit 58B6. The first display aspect changingunit 58B6 changes the display aspect of the first avatar depending onthe relationship between the size of the first viewpoint video and thesize of the first avatar for the first avatar-containing viewpoint videoinput from the first image quality control unit 58B5.

The first display aspect changing unit 58B6 determines whether or notthe ratio of the size of the first avatar to the size of the firstviewpoint video is equal to or more than the first predetermined valuefor the first avatar-containing viewpoint video. In a case in which theratio of the size of the first avatar to the size of the first viewpointvideo is less than the first predetermined value, the first displayaspect changing unit 58B6 transmits the first avatar-containingviewpoint video input from the first image quality control unit 58B5 tothe first viewpoint video output unit 58B7 as it is. In a case in whichthe ratio of the size of the first avatar to the size of the firstviewpoint video is equal to or more than the first predetermined value,the first display aspect changing unit 58B6 changes the display aspectof the first avatar. Then, the first display aspect changing unit 58B6outputs the first avatar-containing viewpoint video obtained by changingthe display aspect to the first viewpoint video output unit 58B7. Thefirst predetermined value may be a fixed value or a variable value.Examples of the fixed value include a value derived in advance by asensory test and/or a computer simulation as a lower limit value of thesize of the avatar that causes a visual discomfort in a case in whichthe avatar enters the viewpoint video 46. Examples of the variable valueinclude a value that can be changed in response to the instructionreceived by any one of the reception device 52, 76, 106, 152, or 202.

An example of changing the display aspect of the first avatar includeshiding the first avatar. That is, in a case in which the ratio of thesize of the first avatar to the size of the first viewpoint video isequal to or more than the first predetermined value, the first displayaspect changing unit 58B6 deletes the first avatar from the firstavatar-containing viewpoint video to change the display aspect of thefirst avatar-containing viewpoint video. For example, as shown in FIG.28, in a case in which the ratio of the size of the first avatar to thesize of the first viewpoint video is equal to or more than the firstpredetermined value in the first avatar-containing viewpoint video, forexample, the first avatar is deleted from the first avatar-containingviewpoint video as shown in FIG. 29. As a result, the display 154 of thefirst HMD 34A displays, for example, the first viewpoint video shown inFIG. 29, that is, the first viewpoint video in a state in which thefirst avatar is hidden.

For example, as shown in FIG. 30, the second image quality control unit58D5 outputs the second avatar-containing viewpoint video to the seconddisplay aspect changing unit 58D6. The second display aspect changingunit 58D6 changes the display aspect of the second avatar depending onthe relationship between the size of the second viewpoint video and thesize of the second avatar for the second avatar-containing viewpointvideo input from the second image quality control unit 58D5.

The second display aspect changing unit 58D6 determines whether or notthe ratio of the size of the second avatar to the size of the secondviewpoint video is equal to or more than the second predetermined value.In a case in which the ratio of the size of the second avatar to thesize of the second viewpoint video is less than the second predeterminedvalue, the second display aspect changing unit 58D6 transmits the secondavatar-containing viewpoint video input from the second image qualitycontrol unit 58D5 to the second viewpoint video output unit 58D7 as itis. In a case in which the ratio of the size of the second avatar to thesize of the second viewpoint video is equal to or more than the secondpredetermined value, the second display aspect changing unit 58D6changes the display aspect of the second avatar. Then, the seconddisplay aspect changing unit 58D6 outputs the second avatar-containingviewpoint video obtained by changing the display aspect to the secondviewpoint video output unit 58D7.

The second predetermined value may be the same value as or a differentvalue from the first predetermined value. In addition, the secondpredetermined value may be a fixed value or a variable value as in thefirst predetermined value.

An example of changing the display aspect of the second avatar includeshiding the second avatar. That is, in a case in which the ratio of thesize of the second avatar to the size of the second viewpoint video isequal to or more than the second predetermined value, the second displayaspect changing unit 58D6 deletes the second avatar from the secondavatar-containing viewpoint video to change the display aspect of thesecond avatar-containing viewpoint video. That is, in a case in whichthe ratio of the size of the second avatar to the size of the secondviewpoint video is equal to or more than the second predetermined valuein the second avatar-containing viewpoint video, the second avatar isdeleted from the second avatar-containing viewpoint video. As a result,the display 204 of the second HMD 34B displays the second viewpointvideo in a state in which the second avatar is hidden.

For example, as shown in FIG. 31, the first avatar-containing viewpointvideo or the first viewpoint video is input from the first displayaspect changing unit 58B6 to the first viewpoint video output unit 58B7.In a case in which the first avatar-containing viewpoint video is inputfrom the first display aspect changing unit 58B6, the first viewpointvideo output unit 58B7 outputs the input first avatar-containingviewpoint video to the first HMD 34A. The first viewpoint video outputunit 58B7 outputs the first avatar-containing viewpoint video to thefirst HMD 34A, and thus the first avatar-containing viewpoint video isdisplayed on the display 154 of the first HMD 34A.

In a case in which the first viewpoint video is input from the firstdisplay aspect changing unit 58B6, the first viewpoint video output unit58B7 outputs the input first viewpoint video to the first HMD 34A. Thefirst viewpoint video output unit 58B7 outputs the first viewpoint videoto the first HMD 34A, and thus the first viewpoint video is displayed onthe display 154 of the first HMD 34A.

For example, as shown in FIG. 32, the second avatar-containing viewpointvideo or the second viewpoint video is input from the second displayaspect changing unit 58D6 to the second viewpoint video output unit58D7. In a case in which the second avatar-containing viewpoint video isinput from the second display aspect changing unit 58D6, the secondviewpoint video output unit 58D7 outputs the input secondavatar-containing viewpoint video to the second HMD 34B. The secondviewpoint video output unit 58D7 outputs the second avatar-containingviewpoint video to the second HMD 34B, and thus the secondavatar-containing viewpoint video is displayed on the display 204 of thesecond HMD 34B.

In a case in which the second viewpoint video is input from the seconddisplay aspect changing unit 58D6, the second viewpoint video outputunit 58D7 outputs the input second viewpoint video to the second HMD34B. The second viewpoint video output unit 58D7 outputs the secondviewpoint video to the second HMD 34B, and thus the second viewpointvideo is displayed on the display 204 of the second HMD 34B.

For example, as shown in FIG. 33, the viewer 28A gives an avatar hideinstruction for instructing the first smartphone 14A to hide the avatar.The avatar hide instruction is received by the touch panel 76A of thefirst smartphone 14A. The first smartphone 14A transmits the avatar hideinstruction received by the touch panel 76A to the setting unit 58E ofthe display control device 12.

On the other hand, the viewer 28B gives the avatar hide instruction tothe second smartphone 14B. The avatar hide instruction is received bythe touch panel 106A of the second smartphone 14B. The second smartphone14B transmits the avatar hide instruction received by the touch panel106A to the setting unit 58E of the display control device 12.

The setting unit 58E performs setting for hiding the second avatar in acase in which the avatar hide instruction transmitted from the secondsmartphone 14B is received. In addition, the setting unit 58E performssetting for hiding the first avatar in a case in which the avatar hideinstruction transmitted from the first smartphone 14A is received. Notethat the setting unit 58E is an example of a “first setting unit” and a“second setting unit” according to the technology of the presentdisclosure.

As shown in FIG. 34 For example, in a case in which the avatar hideinstruction transmitted from the first smartphone 14A is received, thesetting unit 58E outputs flag setting instruction information for givingan instruction for turning an avatar hide flag on to the first controlunit 58B. Here, the avatar hide flag refers to a flag for giving aninstruction for hiding the avatar. In a case in which the avatar hideinstruction transmitted from the second smartphone 14B is received, thesetting unit 58E outputs the flag setting instruction information to thesecond control unit 58D. Note that the reception of the avatar hideinstruction transmitted from the first smartphone 14A by the settingunit 58E is an example of a “second predetermined condition” accordingto the technology of the present disclosure, and the reception of theavatar hide instruction transmitted from the second smartphone 14B bythe setting unit 58E is an example of a “first predetermined condition”according to the technology of the present disclosure.

Here, as an example of the “second predetermined condition” according tothe technology of the present disclosure, the reception of the avatarhide instruction transmitted from the first smartphone 14A by thesetting unit 58E is described as an example, but the technology of thepresent disclosure is not limited to this. For example, the setting forhiding the first avatar may be performed on a condition that the avatarhide instruction received by another reception device, such as thereception device 152 of the first HMD 34A is transmitted to the displaycontrol device 12 and the avatar hide instruction is received by thesetting unit 58E of the display control device 12.

In addition, here, as an example of the “first predetermined condition”according to the technology of the present disclosure, the reception ofthe avatar hide instruction transmitted from the second smartphone 14Bby the setting unit 58E is described as an example, but the technologyof the present disclosure is not limited to this. For example, thesetting for hiding the second avatar may be performed on a conditionthat the avatar hide instruction received by another reception device,such as the reception device 202 of the second HMD 34B is transmitted tothe display control device 12 and the avatar hide instruction isreceived by the setting unit 58E of the display control device 12.

The first control unit 58B turns on the avatar hide flag in a case inwhich the flag setting instruction information is input from the settingunit 58E. In a case in which the avatar hide flag is turned on in thefirst control unit 58B, the first avatar-containing viewpoint video isnot generated, and the first viewpoint video output unit 58B7 outputsthe first viewpoint video to the first HMD 34A.

The second control unit 58D turns on the avatar hide flag in a case inwhich the flag setting instruction information is input from the settingunit 58E. In a case in which the avatar hide flag is turned on in thesecond control unit 58D, the second avatar-containing viewpoint video isnot generated, and the second viewpoint video output unit 58D7 outputsthe second viewpoint video to the second HMD 34B.

Next, an operation of the information processing system 10 will bedescribed.

First, an example of a flow of the first display control processexecuted by the CPU 58 of the display control device 12 according to thefirst display control program 60A will be described with reference toFIGS. 35 to 37. Note that, here, the description will be made on thepremise that the plurality of viewpoint videos are generated byexecuting the viewpoint video generation process by the CPU 58, and theviewpoint video identifier, the peculiar viewpoint position information,the peculiar visual line direction information, and the peculiarangle-of-view information are associated with each viewpoint video. Inaddition, here, the description will be made on the premise that thefirst viewpoint visual line instruction is transmitted from the firstsmartphone 14A to the display control device 12.

In the first display control process shown in FIG. 35, first, in stepST10, the first viewpoint video acquisition unit 58B1 determines whetheror not the first viewpoint visual line instruction transmitted from thefirst smartphone 14A is received. In a case in which the first viewpointvisual line instruction transmitted from the first smartphone 14A is notreceived in step ST10, a negative determination is made, and thedetermination in step ST10 is made again. In a case in which the firstviewpoint visual line instruction transmitted from the first smartphone14A is received in step ST10, a positive determination is made, and thefirst display control process proceeds to step ST12.

In step ST12, the first viewpoint video acquisition unit 58B1 acquiresthe viewpoint video 46 associated with the peculiar viewpoint positioninformation and the peculiar visual line direction informationcorresponding to the first viewpoint visual line instruction as thefirst viewpoint video, and acquires the first viewpoint video identifierassociated with the first viewpoint video.

Subsequently, in step ST14, the first acquisition unit 58A acquires thepeculiar viewpoint position information associated with the viewpointvideo 46 specified from the viewpoint video identifier acquired in stepST12 as the second viewpoint position information, and then the firstdisplay control process proceeds to step ST16.

In step ST16, the first acquisition unit 58A acquires the peculiarvisual line direction information associated with the viewpoint video 46specified from the viewpoint video identifier acquired in step ST12 asthe second visual line direction information, and then the first displaycontrol process proceeds to step ST17.

In step ST17, the first acquisition unit 58A acquires the peculiarangle-of-view information associated with the viewpoint video 46specified from the viewpoint video identifier acquired in step ST12, andthen the first display control process proceeds to step ST18.

In step ST18, the first acquisition unit 58A stores the second viewpointposition information and the like in the first storage region 62A, andthen the first display control process proceeds to step ST20. Here, thesecond viewpoint position information and the like refer to theviewpoint video identifier acquired in step ST12, the second viewpointposition information acquired in step ST14, the second visual linedirection information acquired in step ST16, and the peculiarangle-of-view information acquired in step ST17.

In step ST20, the first acquisition unit 58A determines whether or notthe first viewpoint position information and the like are stored in thesecond storage region 62B. Here, the first viewpoint positioninformation and the like refer to the viewpoint video identifier, thefirst viewpoint position information, the first visual line directioninformation, and the peculiar angle-of-view information. In step ST20,in a case in which the first viewpoint position information and the likeare not stored in the second storage region 62B, a negativedetermination is made, and the determination in step ST20 is made again.In step ST20, in a case in which the first viewpoint positioninformation and the like are stored in the second storage region 62B, apositive determination is made, and the first display control processproceeds to step ST22.

In step ST22, the first acquisition unit 58A acquires the firstviewpoint position information and the like from the second storageregion 62B, and then the first display control process proceeds to stepST24.

In step ST24, the first determination unit 58B2 acquires the firstviewpoint video from the first viewpoint video acquisition unit 58B1,and then the first display control process proceeds to step ST26.

In step ST26, the first determination unit 58B2 determines whether ornot the first viewpoint position indicated by the first viewpointposition information acquired in step ST22 is included in the firstviewpoint video acquired in step ST24. In step ST26, in a case in whichthe first viewpoint position is not included in the first viewpointvideo, a negative determination is made, and the process proceeds tostep ST44 shown in FIG. 37. In step ST26, in a case in which the firstviewpoint position is included in the first viewpoint video, a positivedetermination is made, and the first display control process proceeds tostep ST28 shown in FIG. 36.

In step ST28 shown in FIG. 36, the first determination unit 58B2 outputsthe first person presence information to the first composing unit 58B3,and then the first display control process proceeds to step ST30.

In step ST30, the first composing unit 58B3 causes the first acquisitionunit 58A to acquire the first viewpoint position information and thefirst visual line direction information from the second storage region62B. In addition, the first composing unit 58B3 acquires the firstreference avatar group from the storage 60. In addition, the firstcomposing unit 58B3 acquires the first viewpoint video from the firstviewpoint video acquisition unit 58B1. Then, the first composing unit58B3 generates the first avatar by using the first visual line directioninformation and the first reference avatar group. The first composingunit 58B3 specifies the first viewpoint position indicated by the firstviewpoint position information from the first viewpoint video andsuperimposes the first avatar on the first viewpoint video at thespecified first viewpoint position to generate the firstavatar-containing viewpoint video, and then the first display controlprocess proceeds to step ST34.

In step ST34, the first avatar display size changing unit 58B4 changesthe size of the first avatar in the first avatar-containing viewpointvideo to the size depending on the angle of view indicated by thepeculiar angle-of-view information according to the size derivationtable for the avatar in the storage 60, and then the first displaycontrol process proceeds to step ST36.

In step ST36, the first image quality control unit 58B5 determineswhether or not the image quality of the first avatar in the firstavatar-containing viewpoint video in which the size of the first avataris changed to the size depending on the angle of view in step ST34 doesnot match with the image quality of the first viewpoint video. Here,“not match” refers to the fact that the degree of difference between theimage quality of the first viewpoint video and the image quality of thefirst avatar is outside the first predetermined range. In step ST36, ina case in which the image quality of the first avatar matches with theimage quality of the first viewpoint video, a negative determination ismade, and the first display control process proceeds to step ST40. Instep ST36, in a case in which the image quality of the first avatar doesnot match with the image quality of the first viewpoint video, apositive determination is made, and the first display control processproceeds to step ST38.

In step ST38, the first image quality control unit 58B5 controls theimage quality of the first avatar-containing viewpoint video such thatthe degree of difference between the image quality of the firstviewpoint video and the image quality of the first avatar is within thefirst predetermined range in the first avatar-containing viewpointvideo, and then the first display control process proceeds to step ST40.

In step ST40, the first display aspect changing unit 58B6 determineswhether or not a condition of changing the display aspect of the firstavatar included in the first avatar-containing viewpoint video obtainedby controlling the image quality of the first avatar by the first imagequality control unit 58B5 (first display aspect change condition) issatisfied. Here, the first display aspect change condition refers to,for example, a condition that, for the first avatar-containing viewpointvideo obtained by controlling the image quality of the first avatar bythe first image quality control unit 58B5, the ratio of the first avatarto the size of the whole first avatar-containing viewpoint video, thatis, the size of the first viewpoint video is equal to or more than thefirst predetermined value.

In a case in which the first display aspect change condition is notsatisfied in step ST40, a negative determination is made, and the firstdisplay control process proceeds to step ST46 shown in FIG. 37. In acase in which the first display aspect change condition is satisfied instep ST40, a positive determination is made, and the first displaycontrol process proceeds to step ST42.

In step ST42, the first display aspect changing unit 58B6 changes thedisplay aspect of the first avatar-containing viewpoint video bydeleting the first avatar from the first avatar-containing viewpointvideo. By deleting the first avatar from the first avatar-containingviewpoint video, the first avatar-containing viewpoint video is changedto the first viewpoint video. In a case in which the execution of theprocess of step ST42 is terminated, the first display control processproceeds to step ST46 shown in FIG. 37.

In step ST44 shown in FIG. 37, the first viewpoint video output unit58B7 outputs the first viewpoint video to the first HMD 34A. The firstHMD 34A causes the display 154 to display the first viewpoint videoinput from the first viewpoint video output unit 58B7. In a case inwhich the execution of the process of step ST44 is terminated, the firstdisplay control process proceeds to step ST48.

In step ST46 shown in FIG. 37, the first viewpoint video output unit58B7 outputs the first avatar-containing viewpoint video or the firstviewpoint video to the first HMD 34A. That is, the first viewpoint videooutput unit 58B7 outputs the first avatar-containing viewpoint video tothe first HMD 34A in a case in which a negative determination is made instep ST40, and outputs the first viewpoint video to the first HMD 34A ina case in which a positive determination is made in step ST40. The firstHMD 34A causes the display 154 to display the first viewpoint videoinput from the first viewpoint video output unit 58B7, and causes thedisplay 154 to display the first avatar-containing viewpoint video inputfrom the first viewpoint video output unit 58B7. In a case in which theexecution of the process of step ST46 is terminated, the first displaycontrol process proceeds to step ST48.

In step ST48, the CPU 58 determines whether or not a condition forterminating the first display control process (first display controlprocess termination condition) is satisfied. Examples of the firstdisplay control process termination condition include a condition thatan instruction for terminating the first display control process isreceived by the reception device 52, 76, 106, 152 or 202.

In a case in which the first display control process terminationcondition is not satisfied in step ST48, a negative determination ismade, and the first display control process proceeds to step ST10 shownin FIG. 35. In a case in which the first display control processtermination condition is satisfied in step ST48, a positivedetermination is made, and the first display control process isterminated.

Next, an example of a flow of the second display control processexecuted by the CPU 58 of the display control device 12 according to thesecond display control program 60B will be described with reference toFIGS. 38 to 40. Note that, here, the description will be made on thepremise that the plurality of viewpoint videos are generated byexecuting the viewpoint video generation process by the CPU 58, and theviewpoint video identifier, the peculiar viewpoint position information,the peculiar visual line direction information, and the peculiarangle-of-view information are associated with each viewpoint video. Inaddition, here, the description will be made on the premise that thesecond viewpoint visual line instruction is transmitted from the secondsmartphone 14B to the display control device 12.

In the second display control process shown in FIG. 38, first, in stepST100, the second viewpoint video acquisition unit 58D1 determineswhether or not the second viewpoint visual line instruction transmittedfrom the second smartphone 14B is received. In a case in which thesecond viewpoint visual line instruction transmitted from the secondsmartphone 14B is not received in step ST100, a negative determinationis made, and the determination in step ST100 is made again. In a case inwhich the second viewpoint visual line instruction transmitted from thesecond smartphone 14B is received in step ST100, a positivedetermination is made, and the second display control process proceedsto step ST102.

In step ST102, the second viewpoint video acquisition unit 58D1 acquiresthe viewpoint video 46 associated with the peculiar viewpoint positioninformation and the peculiar visual line direction informationcorresponding to the second viewpoint visual line instruction as thesecond viewpoint video, and acquires the second viewpoint videoidentifier associated with the second viewpoint video.

Subsequently, in step ST104, the second acquisition unit 58C acquiresthe peculiar viewpoint position information associated with theviewpoint video 46 specified from the viewpoint video identifieracquired in step ST102 as the first viewpoint position information, andthen the second display control process proceeds to step ST106.

In step ST106, the second acquisition unit 58C acquires the peculiarvisual line direction information associated with the viewpoint video 46specified from the viewpoint video identifier acquired in step ST102 asthe first visual line direction information, and then the second displaycontrol process proceeds to step ST107.

In step ST107, the second acquisition unit 58C acquires the peculiarangle-of-view information associated with the viewpoint video 46specified from the viewpoint video identifier acquired in step ST102,and then the second display control process proceeds to step ST108.

In step ST108, the second acquisition unit 58C stores the firstviewpoint position information and the like in the second storage region62B, and then the second display control process proceeds to step ST110.Here, the first viewpoint position information and the like refer to theviewpoint video identifier acquired in step ST102, the first viewpointposition information acquired in step ST104, the first visual linedirection information acquired in step ST106, and the peculiarangle-of-view information acquired in step ST107.

In step ST110, the second acquisition unit 58C determines whether or notthe second viewpoint position information and the like are stored in thefirst storage region 62A. Here, the second viewpoint positioninformation and the like refer to the viewpoint video identifier, thesecond viewpoint position information, the second visual line directioninformation, and the peculiar angle-of-view information. In step ST110,in a case in which the second viewpoint position information and thelike are not stored in the first storage region 62A, a negativedetermination is made, and the determination in step ST110 is madeagain. In step ST110, in a case in which the second viewpoint positioninformation and the like are stored in the first storage region 62A, apositive determination is made, and the second display control processproceeds to step ST112.

In step ST112, the second acquisition unit 58C acquires the secondviewpoint position information and the like from the first storageregion 62A, and then the second display control process proceeds to stepST114.

In step ST114, the second determination unit 58D2 acquires the secondviewpoint video from the second viewpoint video acquisition unit 58D1,and then the second display control process proceeds to step ST116.

In step ST116, the second determination unit 58D2 determines whether ornot the second viewpoint position indicated by the second viewpointposition information acquired in step ST112 is included in the secondviewpoint video acquired in step ST114. In step ST116, in a case inwhich the second viewpoint position is not included in the secondviewpoint video, a negative determination is made, and the seconddisplay control process proceeds to step ST134 shown in FIG. 40. In stepST116, in a case in which the second viewpoint position is included inthe second viewpoint video, a positive determination is made, and thesecond display control process proceeds to step ST118 shown in FIG. 39.

In step ST118 shown in FIG. 39, the second determination unit 58D2outputs the second person presence information to the second composingunit 58D3, and then the second display control process proceeds to stepST120.

In step ST120, the second composing unit 58D3 causes the secondacquisition unit 58C to acquire the second viewpoint positioninformation and the second visual line direction information from thefirst storage region 62A. In addition, the second composing unit 58D3acquires the second reference avatar group from the storage 60. Inaddition, the second composing unit 58D3 acquires the second viewpointvideo from the second viewpoint video acquisition unit 58D1. Then, thesecond composing unit 58D3 generates the second avatar by using thesecond visual line direction information and the second reference avatargroup. The second composing unit 58D3 specifies the second viewpointposition indicated by the second viewpoint position information from thesecond viewpoint video and superimposes the second avatar on the secondviewpoint video at the specified second viewpoint position to generatethe second avatar-containing viewpoint video, and then the seconddisplay control process proceeds to step ST124.

In step ST124, the second avatar display size changing unit 58D4 changesthe size of the second avatar in the second avatar-containing viewpointvideo to the size depending on the angle of view indicated by thepeculiar angle-of-view information according to the size derivationtable for the avatar in the storage 60, and then the second displaycontrol process proceeds to step ST126.

In step ST126, the second image quality control unit 58D5 determineswhether or not the image quality of the second avatar in the secondavatar-containing viewpoint video in which the size of the second avataris changed to the size depending on the angle of view in step ST124 doesnot match with the image quality of the second viewpoint video. Here,“not match” refers to the fact that the degree of difference between theimage quality of the second viewpoint video and the image quality of thesecond avatar is outside the second predetermined range. In step ST126,in a case in which the image quality of the second avatar matches withthe image quality of the second viewpoint video, a negativedetermination is made, and the second display control process proceedsto step ST130. In step ST126, in a case in which the image quality ofthe second avatar does not match with the image quality of the secondviewpoint video, a positive determination is made, and the seconddisplay control process proceeds to step ST128.

In step ST128, the second image quality control unit 58D5 controls theimage quality of the second avatar-containing viewpoint video such thatthe degree of difference between the image quality of the secondviewpoint video and the image quality of the second avatar is within thesecond predetermined range in the second avatar-containing viewpointvideo, and then the second display control process proceeds to stepST130.

In step ST130, the second display aspect changing unit 58D6 determineswhether or not a condition of changing the display aspect of the secondavatar included in the second avatar-containing viewpoint video obtainedby controlling the image quality of the second avatar by the secondimage quality control unit 58D5 (second display aspect change condition)is satisfied. Here, the second display aspect change condition refersto, for example, a condition that, for the second avatar-containingviewpoint video obtained by controlling the image quality of the secondavatar by the second image quality control unit 58D5, the ratio of thesecond avatar to the size of the whole second avatar-containingviewpoint video, that is, the size of the second viewpoint video isequal to or more than the second predetermined value.

In a case in which the second display aspect change condition is notsatisfied in step ST130, a negative determination is made, and thesecond display control process proceeds to step ST136 shown in FIG. 40.In a case in which the second display aspect change condition issatisfied in step ST130, a positive determination is made, and thesecond display control process proceeds to step ST132.

In step ST132, the second display aspect changing unit 58D6 changes thedisplay aspect of the second avatar-containing viewpoint video bydeleting the second avatar from the second avatar-containing viewpointvideo. By deleting the second avatar from the second avatar-containingviewpoint video, the second avatar-containing viewpoint video is changedto the second viewpoint video. In a case in which the execution of theprocess of step ST132 is terminated, the second display control processproceeds to step ST136 shown in FIG. 40.

In step ST134 shown in FIG. 40, the second viewpoint video output unit58D7 outputs the second viewpoint video to the second HMD 34B. Thesecond HMD 34B causes the display 204 to display the second viewpointvideo input from the second viewpoint video output unit 58D7. In a casein which the execution of the process of step ST134 is terminated, thesecond display control process proceeds to step ST138.

In step ST136 shown in FIG. 40, the second viewpoint video output unit58D7 outputs the second avatar-containing viewpoint video or the secondviewpoint video to the second HMD 34B. That is, the second viewpointvideo output unit 58D7 outputs the second avatar-containing viewpointvideo to the second HMD 34B in a case in which a negative determinationis made in step ST130, and outputs the second viewpoint video to thesecond HMD 34B in a case in which a positive determination is made instep ST130. The second HMD 34B causes the display 204 to display thesecond viewpoint video input from the second viewpoint video output unit58D7, and causes the display 204 to display the second avatar-containingviewpoint video input from the second viewpoint video output unit 58D7.In a case in which the execution of the process of step ST136 isterminated, the second display control process proceeds to step ST138.

In step ST138, the CPU 58 determines whether or not a condition forterminating the second display control process (second display controlprocess termination condition) is satisfied. Examples of the seconddisplay control process termination condition include a condition thatan instruction for terminating the second display control process isreceived by the reception device 52, 76, 106, 152, or 202.

In a case in which the second display control process terminationcondition is not satisfied in step ST138, a negative determination ismade, and the second display control process proceeds to step ST100shown in FIG. 38. In a case in which the second display control processtermination condition is satisfied in step ST138, a positivedetermination is made, and the second display control process isterminated.

Next, an example of a flow of the setting process executed by the CPU 58of the display control device 12 according to the setting program 60Cwill be described with reference to FIG. 41. Note that, here, thedescription will be made on the premise that the avatar hide instructionis transmitted from the first smartphone 14A. In addition, the settingprocess in a case in which the avatar hide instruction is transmittedfrom the second smartphone 14B is different from the setting process ina case in which the avatar hide instruction is transmitted from thefirst smartphone 14A only in that the second control unit 58D isoperated by the setting unit 58E operating the second control unit 58D.Therefore, the description of the setting process in a case in which theavatar hide instruction is transmitted from the second smartphone 14Bwill be omitted.

In the setting process shown in FIG. 41, first, in step ST200, thesetting unit 58E determines whether or not the avatar hide instructiontransmitted from the first smartphone 14A is received. In a case inwhich the avatar hide instruction transmitted from the first smartphone14A is not received in step ST200, a negative determination is made, andthe determination in step ST200 is made again. In a case in which theavatar hide instruction transmitted from the first smartphone 14A isreceived in step ST200, a positive determination is made, and thesetting process proceeds to step ST202.

In step ST202, the setting unit 58E outputs the flag setting instructioninformation to the first control unit 58B, and then the setting processproceeds to step ST204.

In step ST204, the first control unit 58B determines whether or not theavatar hide flag is turned off. In a case in which the avatar hide flagis turned on in step ST204, a negative determination is made, and thesetting process proceeds to step ST208. In a case in which the avatarhide flag is turned off in step ST204, a positive determination is made,and the setting process proceeds to step ST206.

In step ST206, the first control unit 58B changes the avatar hide flagfrom off to on, and then the setting process proceeds to step ST208. Ina case in which the avatar hide flag is turned on, the firstavatar-containing viewpoint video is not generated, and the firstviewpoint video is displayed on the display 154 of the first HMD 34A.

In step ST208, the setting unit 58E determines whether or not acondition for terminating the setting process (setting processtermination condition) is satisfied. Examples of the setting processtermination condition include a condition that the instruction forterminating the setting process is received by any one of the receptiondevices 52, 76, 106, 152, or 202.

In a case in which the setting process termination condition is notsatisfied in step ST208, a negative determination is made, and thesetting process proceeds to step ST200. In a case in which the settingprocess termination condition is satisfied in step ST208, a positivedetermination is made, and the setting process is terminated.

As described above, in the display control device 12, the first avatarfor specifying the first viewpoint position is displayed in the firstviewpoint video in a case in which the first viewpoint position isincluded in the first viewpoint video selected from among the pluralityof viewpoint videos 46. Then, the size of the first avatar is changeddepending on the angle of view of the first viewpoint video, so that thedisplay size of the first avatar is changed depending on the angle ofview of the first viewpoint video. Therefore, in a state in which thepresence of the viewer 28B can be perceived through the first viewpointvideo selected from among the plurality of viewpoint videos 46, a senseof presence of the viewer 28B can be changed depending on the angle ofview of the viewpoint video viewed by the viewer 28A.

In addition, the display control device 12 performs a control of settingthe degree of difference between the image quality of the firstviewpoint video and the image quality of the first avatar within thefirst predetermined range. Therefore, as compared to a case in which thedegree of difference between the image quality of the first viewpointvideo and the image quality of the first avatar is outside the firstpredetermined range, a visual discomfort caused due to the difference ofthe image quality of the first viewpoint video and the image quality ofthe first avatar can be reduced.

In addition, in the display control device 12, the display aspect of thefirst avatar is changed depending on the relationship between the sizeof the first viewpoint video and the size of the first avatar, that is,the relationship between the display size of the first viewpoint videoand the display size of the first avatar. Therefore, as compared to acase in which the display aspect of the first avatar is fixed regardlessof the relationship between the display size of the first viewpointvideo and the display size of the first avatar, a visual discomfortcaused due to the difference of the display size of the first viewpointvideo and the display size of the first avatar can be reduced.

In addition, in the display control device 12, the display aspect of thefirst avatar is changed in a case in which the ratio of the display sizeof the first avatar to the display size of the first viewpoint video isequal to or more than the first predetermined value. Specifically, thefirst avatar is hidden. Therefore, as compared to a case in which thedisplay aspect of the first avatar is fixed regardless of whether or notthe ratio of the display size of the first avatar to the display size ofthe first viewpoint video is equal to or more than the firstpredetermined value, a visual discomfort caused by the presence of thefirst avatar prohibiting the visual recognition of the first viewpointvideo can be reduced.

In addition, in the display control device 12, the first HMD 34A havingthe display 154 is mounted on the head of the viewer 28A, and the firstavatar-containing viewpoint video is displayed on the display 154.Therefore, the first viewpoint video and the first avatar can bevisually perceived by the viewer 28A through the first HMD 34A.

In addition, in the display control device 12, the first viewpoint videois selected from among the plurality of viewpoint videos 46 in responseto the first viewpoint visual line instruction received by the touchpanel 76A of the first smartphone 14A. Therefore, it is possible toprovide the viewpoint video from among the plurality of viewpoint videos46, which is intended by the viewer 28A, to the viewer 28A.

In addition, in the display control device 12, the first visual linedirection information is acquired by the first acquisition unit 58A.Then, the avatar for specifying the first visual line directionindicated by the first visual line direction information acquired by thefirst acquisition unit 58A is generated as the first avatar. Therefore,the visual line direction of the viewer 28B can be perceived by theviewer 28A through the first viewpoint video selected from among theplurality of viewpoint videos 46.

In addition, in the display control device 12, each of the plurality ofviewpoint videos 46 has the peculiar viewpoint position information. Inaddition, each of the plurality of viewpoint videos 46 is the videoshowing the imaging region observed from the viewpoint positionindicated by the corresponding peculiar viewpoint position information.Then, the peculiar viewpoint position information of any one of theplurality of viewpoint videos 46 is acquired by the first acquisitionunit 58A as the first viewpoint position information. Therefore, ascompared to a case in which the viewpoint position of the viewer 28A isdetermined independently of the viewpoint video 46, the viewpointposition having a strong association with the viewpoint video 46 can bedetermined as the viewpoint position of the viewer 28A.

In addition, in the display control device 12, the peculiar viewpointposition information corresponding to the second viewpoint videodisplayed on the display 204 of the second HMD 34B is acquired by thefirst acquisition unit 58A as the first viewpoint position information.Therefore, as compared to a case in which the viewpoint position of theviewer 28A is determined independently of the second viewpoint videodisplayed on the display 204 of the second HMD 34B, the viewpointposition having a strong association with the second viewpoint videodisplayed on the display 204 of the second HMD 34B can be determined asthe viewpoint position of the viewer 28A.

In addition, in the display control device 12, the informationindicating the direction facing the second viewpoint video displayed onthe display 204 of the second HMD 34B is acquired by the firstacquisition unit 58A as the first visual line direction information.Therefore, the visual line direction of the viewer 28B can be easilydetermined as compared to a case in which the visual line direction ofthe viewer 28B is determined by detecting the visual line direction ofthe viewer 28B by a detection device.

In addition, in the display control device 12, the second HMD 34B havingthe display 204 is mounted on the head of the viewer 28B, and the secondavatar-containing viewpoint video is displayed on the display 204.Therefore, the second viewpoint video and the second avatar can bevisually perceived by the viewer 28B through the second HMD 34B.

In addition, in the display control device 12, the second viewpointvideo is selected from among the plurality of viewpoint videos 46 inresponse to the second viewpoint visual line instruction received by thetouch panel 106A of the second smartphone 14B. Therefore, it is possibleto provide the viewpoint video from among the plurality of viewpointvideos 46, which is intended by the viewer 28B, to the viewer 28B.

In addition, in the display control device 12, the second avatar forspecifying the second viewpoint position is displayed in the secondviewpoint video in a case in which the second viewpoint position isincluded in the second viewpoint video selected from among the pluralityof viewpoint videos 46. Then, the size of the second avatar is changeddepending on the angle of view of the second viewpoint video, so thatthe display size of the second avatar is changed depending on the angleof view of the second viewpoint video. Therefore, in a state in whichthe viewers 28A and 28B can perceive each other's presence through theviewpoint video 46 selected from among the plurality of viewpoint videos46, the senses of presence of the viewers 28A and 28B can be changeddepending on the angle of view of the viewpoint video 46 visuallyrecognized.

In addition, the display control device 12 performs a control of settingthe degree of difference between the image quality of the secondviewpoint video and the image quality of the second avatar within thesecond predetermined range. Therefore, as compared to a case in whichthe degree of difference between the image quality of the secondviewpoint video and the image quality of the second avatar is outsidethe second predetermined range, a visual discomfort caused due to thedifference of the image quality of the second viewpoint video and theimage quality of the second avatar can be reduced.

In addition, in the display control device 12, the display aspect of thefirst avatar is changed depending on the relationship between the sizeof the second viewpoint video and the size of the second avatar, thatis, the relationship between the display size of the second viewpointvideo and the display size of the second avatar. Therefore, as comparedto a case in which the display aspect of the second avatar is fixedregardless of the relationship between the display size of the secondviewpoint video and the display size of the second avatar, a visualdiscomfort caused due to the difference of the display size of thesecond viewpoint video and the display size of the second avatar can bereduced.

In addition, in the display control device 12, the display aspect of thesecond avatar is changed in a case in which the ratio of the displaysize of the second avatar to the display size of the second viewpointvideo is equal to or more than the second predetermined value.Specifically, the second avatar is hidden. Therefore, as compared to acase in which the display aspect of the second avatar is fixedregardless of whether or not the ratio of the display size of the secondavatar to the display size of the second viewpoint video is equal to ormore than the second predetermined value, a visual discomfort caused bythe presence of the second avatar prohibiting the visual recognition ofthe second viewpoint video can be reduced.

In addition, in the display control device 12, the second visual linedirection information is acquired by the second acquisition unit 58C.Then, the avatar for specifying the second visual line directionindicated by the second visual line direction information acquired bythe second acquisition unit 58C is generated as the second avatar.Therefore, the visual line direction of the viewer 28A can be perceivedby the viewer 28B through the second viewpoint video selected from amongthe plurality of viewpoint videos 46.

In addition, in the display control device 12, each of the plurality ofviewpoint videos 46 has the peculiar viewpoint position information. Inaddition, each of the plurality of viewpoint videos 46 is the videoshowing the imaging region observed from the viewpoint positionindicated by the corresponding peculiar viewpoint position information.Then, the peculiar viewpoint position information of any one of theplurality of viewpoint videos 46 is acquired by the first acquisitionunit 58A as the first viewpoint position information, and the peculiarviewpoint position information of any one of the plurality of viewpointvideos 46 is acquired by the second acquisition unit 58C as the secondviewpoint position information. Therefore, as compared to a case inwhich the viewpoint position of each of the viewers 28A and 28B isdetermined independently of the viewpoint video 46, the viewpointposition having a strong association with the viewpoint video 46 can bedetermined as the viewpoint position of each of the viewers 28A and 28B.

In addition, in the display control device 12, the peculiar viewpointposition information corresponding to the second viewpoint videodisplayed on the display 204 of the second HMD 34B is acquired by thefirst acquisition unit 58A as the first viewpoint position information.In addition, the peculiar viewpoint position information correspondingto the first viewpoint video displayed on the display 154 of the firstHMD 34A is acquired by the second acquisition unit 58C as the secondviewpoint position information. Therefore, as compared to a case inwhich the viewpoint position of each of the viewers 28A and 28B isdetermined independently of the displayed viewpoint video 46, theviewpoint position having a strong association with the displayedviewpoint video 46 can be determined as the viewpoint position of eachof the viewers 28A and 28B.

In addition, in the display control device 12, the informationindicating the direction facing the second viewpoint video displayed onthe display 204 of the second HMD 34B is acquired by the firstacquisition unit 58A as the first visual line direction information. Inaddition, the information indicating the direction facing the firstviewpoint video displayed on the display 154 of the first HMD 34A isacquired by the second acquisition unit 58C as the second visual linedirection information. Therefore, as compared to a case in which thevisual line direction of the viewer 28B is determined by detecting thevisual line direction of the viewer 28B by a first detection device andthe visual line direction of the viewer 28A is determined by detectingthe visual line direction of the viewer 28A by a second detectiondevice, the visual line direction of each of the viewers 28A and 28B canbe easily determined.

In addition, in the display control device 12, the setting unit 58Eperforms setting for hiding the second avatar in a case in which theavatar hide instruction transmitted from the second smartphone 14B isreceived by the setting unit 58E. Therefore, it is possible to hide thesecond avatar in the viewpoint video 46 depending on the intention ofthe viewer 28B.

In addition, in the display control device 12, the first viewpointposition and the second viewpoint position are limited to a partialregion of the imaging region (in the examples shown in FIGS. 1 and 3,the spectator seat 26). Therefore, as compared to a case in which thefirst viewpoint position and the second viewpoint position are notlimited to a partial region of the imaging region, the viewers 28A and28B can easily perceive each other's presence through the viewpointvideo selected from among the plurality of viewpoint videos 46.

In addition, in the display control device 12, the setting unit 58Eperforms setting for hiding the first avatar in a case in which theavatar hide instruction transmitted from the first smartphone 14A isreceived by the setting unit 58E. Therefore, it is possible to hide thefirst avatar in the viewpoint video 46 depending on the intention of theviewer 28A.

Further, in the display control device 12, the virtual viewpoint video46C is included in the plurality of viewpoint videos 46. Therefore, itis possible to grasp the aspect of the imaging region observed from aposition at which the actual imaging apparatus is not present.

Note that, in the embodiment described above, the first viewpointposition and the second viewpoint position are limited to a partialregion of the imaging region (in the examples shown in FIGS. 1 and 3,the spectator seat 26), but the technology of the present disclosure isnot limited to this, and the first viewpoint position or the secondviewpoint position may be limited to a partial region of the imagingregion.

In addition, in the embodiment described above, the hide of the avatarhas been described as an example of changing the display aspect of theavatar, but the technology of the present disclosure is not limited tothis. For example, another example of changing the display aspect of theavatar includes display of only the outline of the avatar, andtranslucency of the avatar.

In addition, in the embodiment described above, the aspect example hasbeen described in which the display aspect of the first avatar ischanged only depending on the relationship between the display size ofthe first viewpoint video and the display size of the first avatar, butthe technology of the present disclosure is not limited to this. Forexample, the first display aspect changing unit 58B6 may change thedisplay aspect of the first avatar depending on the relationship betweenthe display size of the first viewpoint video and the display size ofthe first avatar and the relationship between the display position ofthe first viewpoint video and the display position of the first avatar.

For example, as shown in FIG. 42, in a case in which the ratio of thedisplay size of the first avatar to the display size of the firstavatar-containing viewpoint video is equal to or more than the firstpredetermined value and the first avatar is superimposed on a specificregion (center region 180 in the example shown in FIG. 42) of the firstavatar-containing viewpoint video, the display aspect of the firstavatar is changed by the first display aspect changing unit 58B6.Examples of the change in the display aspect of the first avatar includedeleting the first avatar from the first avatar-containing viewpointvideo, displaying only the outline of the first avatar, and translucencythe first avatar.

With the present configuration, as compared to the display aspect of thefirst avatar is fixed regardless of the relationship between the displaysize of the first viewpoint video and the display size of the firstavatar and the relationship between the display position of the firstviewpoint video and the display position of the first avatar, a visualdiscomfort caused by the difference between the display size of thefirst viewpoint video and the display size of the first avatar and thedifference between the display position of the first viewpoint video andthe display position of the first avatar can be reduced.

In the example shown in FIG. 42, the display aspect of the first avataris changed in a case in which the first avatar is superimposed on thecenter region 180, but the technology of the present disclosure is notlimited to this, and the display aspect of the first avatar may bechanged in a case in which the first avatar is superimposed on a regiondifferent from the center region 180 of the first avatar-containingviewpoint video.

In addition, in the example shown in FIG. 42, the relationships of thesize and the position between the first avatar-containing viewpointvideo and the first avatar are described as an example. However, thedisplay aspect of the second avatar may be changed by the second displayaspect changing unit 58D6 in the same manner as the change of thedisplay aspect of the first avatar as long as the same conditions aresatisfied for the relationships of the size and the position between thesecond avatar-containing viewpoint video and the second avatar. As aresult, as compared to the display aspect of the second avatar is fixedregardless of the relationship between the display size of the secondviewpoint video and the display size of the second avatar and therelationship between the display position of the second viewpoint videoand the display position of the second avatar, a visual discomfortcaused by the difference between the display size of the secondviewpoint video and the display size of the second avatar and thedifference between the display position of the second viewpoint videoand the display position of the second avatar can be reduced.

In addition, in the embodiment described above, the aspect example hasbeen described in which the display aspect of the first avatar ischanged by hiding the first avatar by the first display aspect changingunit 58B6 in a case in which the ratio of the display size of the firstavatar to the display size of the first viewpoint video is equal to ormore than the first predetermined value, but the technology of thepresent disclosure is not limited to this. For example, in a case inwhich the ratio of the display size of the first avatar to the displaysize of the first viewpoint video is less than a third predeterminedvalue, the first display aspect changing unit 58B6 may cause the display154 of the first HMD 34A to display the first avatar in the displayaspect emphasized more than other regions of the first avatar-containingviewpoint video. Examples of the display aspect emphasized more thanother regions include highlighting the outline of the first avatar,displaying with a mark, such as an arrow indicating the visual linedirection of the first avatar, and pop-up display of the first avatar.Note that the third predetermined value is an example of a “secondthreshold value” according to the technology of the present disclosure.Examples of the third predetermined value include the firstpredetermined value described above. In addition, the thirdpredetermined value may be a value smaller than the first predeterminedvalue.

With the present configuration, as compared to a case in which thedisplay aspect of the first avatar is fixed regardless of whether or notthe ratio of the display size of the first avatar to the display size ofthe first viewpoint video is less than the third predetermined value,the presence of the viewer 28B can easily be perceived through theviewpoint video selected from among the plurality of viewpoint videos46.

Here, the aspect example has been described in which the first avatar isdisplayed in the display aspect emphasized more than other regions ofthe first avatar-containing viewpoint video in a case in which the ratioof the display size of the first avatar to the display size of the firstviewpoint video is less than the third predetermined value, but thetechnology of the present disclosure is not limited to this. Forexample, the display aspect of the second avatar may be changed by thesecond display aspect changing unit 58D6 in the same manner as thechange of the display aspect of the first avatar as long as the samecondition is satisfied for the relationship between the display size ofthe second viewpoint video and the display size of the second avatar. Asa result, as compared to a case in which the display aspect of thesecond avatar is fixed regardless of whether or not the ratio of thedisplay size of the second avatar to the display size of the secondviewpoint video is less than the third predetermined value, the presenceof the viewer 28A can easily be perceived through the viewpoint videoselected from among the plurality of viewpoint videos 46. Note that, inthis case, the “third predetermined value” is an example of a “fourththreshold value” according to the technology of the present disclosure.

In addition, in the example shown in FIG. 42, the aspect example hasbeen described in which the display aspect is changed in a case in whichthe first avatar is superimposed on the center region 180, but thetechnology of the present disclosure is not limited to this. Forexample, as shown in FIGS. 43 and 44, the display aspect may be changedby the first display aspect changing unit 58B6 in a case in which atleast a part of the first avatar is superimposed on a region-of-interestof the viewer 28A with respect to the first avatar-containing viewpointvideo. In the example shown in FIG. 43, the first avatar is notsuperimposed on the region-of-interest, whereas in the example shown inFIG. 44, the first avatar is superimposed on the region-of-interest. Inthis case, a translucent first avatar that the first avatar istranslucent is superimposed and displayed on the first viewpoint video.

The region-of-interest may be determined in response to the instructionreceived by the reception device 76 of the first smartphone 14A and/orthe reception device 152 of the first HMD 34A. In addition, an eyemovement of the viewer 28A may be detected by the eye tracker 166 (seeFIG. 6) of the first HMD 34A, and the region-of-interest in the firstavatar-containing viewpoint video may be determined depending on thedetection result.

Note that the display aspect may be changed by the second display aspectchanging unit 58D6 in a case in which at least a part of the secondavatar is superimposed on a region-of-interest of the viewer 28B withrespect to the second avatar-containing viewpoint video. In this case aswell, the region-of-interest may be determined in response to theinstruction received by the reception device 106 of the secondsmartphone 14B and/or the reception device 202 of the second HMD 34B. Inaddition, an eye movement of the viewer 28B may be detected by the eyetracker 216 (see FIG. 6) of the second HMD 34B, and theregion-of-interest in the second avatar-containing viewpoint video maybe determined depending on the detection result.

In addition, in the embodiment described above, the aspect example hasbeen described in which the peculiar visual line direction informationis used as the first visual line direction information and the secondvisual line direction information, but the technology of the presentdisclosure is not limited to this. For example, the first visual linedirection information may be determined based on the detection result ofthe eye tracker 166, or the second visual line direction information maybe determined based on the detection result of the eye tracker 216.

In addition, in the embodiment described above, the description has beenmade on the premise that neither the viewers 28A nor 28B spectates inthe spectator seat 26, but the viewer 28A and/or the viewer 28B mayspectate in the spectator seat 26. In this case, the positions of theviewers 28A and 28B may be specified based on the reception results ofthe GPS receivers 72 and 102, and the specified positions of the viewers28A and 28B may be used as the first viewpoint position and the secondviewpoint position.

In addition, in the embodiment described above, the aspect example hasbeen described in which one viewpoint video is selected as the firstviewpoint video from among the plurality of viewpoint videos 46 inresponse to the first viewpoint visual line instruction given to thefirst smartphone 14A, but the technology of the present disclosure isnot limited to this. For example, in a state in which the plurality ofviewpoint videos 46 are displayed on the display 78 of the firstsmartphone 14A, the viewer 28A may select any of the viewpoint videos 46as the first viewpoint video via the touch panel 76A. Note that the sameapplies to the selection of the second viewpoint video.

In addition, in the embodiment described above, the avatar has beendescribed as an example, but the technology of the present disclosure isnot limited to this, and any information may be adopted as long as oneof the viewers 28A and 28B can specify the viewpoint position and thevisual line direction of the other thereof. Examples of the informationfor specifying the viewpoint position and the visual line directioninclude the mark, such as the arrow, a combination of the mark, such asan arrow, and the avatar, or an arrow indicating the position of theavatar.

In addition, in the embodiment described above, the aspect example hasbeen described in which the first display control process, the seconddisplay control process, and the setting process (hereinafter referredto as a “display control device side process” in a case in which adistinction is not necessary) are executed by the CPU 58 of the displaycontrol device 12, but the technology of the present disclosure is notlimited to this, and the display control device side process may bedistributed and executed by the first smartphone 14A, the secondsmartphone 14B, the first HMD 34A, and the second HMD 34B.

For example, the first display control process and the setting processmay be executed by the first HMD 34A, and the second display controlprocess and the setting process may be executed by the second HMD 34B.In this case, as shown in FIG. 45, for example, the first displaycontrol program 60A and the setting program 60C are stored in thestorage 162 of the first HMD 34A. The CPU 160 executes the first displaycontrol process by being operated as the first acquisition unit 58A andthe first control unit 58B according to the first display controlprogram 60A. In addition, the CPU 160 executes the setting process bybeing operated as the setting unit 58E according to the setting program60C.

On the other hand, as shown in FIG. 46, for example, the second displaycontrol program 60B and the setting program 60C are stored in thestorage 212 of the second HMD 34B. The CPU 210 executes the seconddisplay control process by being operated as the second acquisition unit58C and the second control unit 58D according to the second displaycontrol program 60B. In addition, the CPU 210 executes the settingprocess by being operated as the setting unit 58E according to thesetting program 60C.

In addition, in the embodiment described above, the first HMD 34A andthe second HMD 34B have been described as examples, but the technologyof the present disclosure is not limited to this, and at least one ofthe first HMD 34A or the second HMD 34B can be substituted with variousdevices equipped with an arithmetic device, such as a smartphone, atablet terminal, a head-up display, or a personal computer.

In addition, in the embodiment described above, the soccer stadium 22has been described as an example, but it is merely an example, and anyplace, such as a baseball stadium, a curling stadium, and a swimmingpool, may be adopted as long as the plurality of imaging apparatuses canbe installed.

In addition, in the embodiment described above, the wirelesscommunication method using the base station 20 has been described as anexample, but it is merely an example, and the technology of the presentdisclosure is established even in the wired communication method usingthe cable.

In addition, in the embodiment described above, the unmanned aerialvehicle 27 has been described as an example, but the technology of thepresent disclosure is not limited to this, and the imaging region may beimaged by the imaging apparatus 18 suspended by a wire (for example, aself-propelled imaging apparatus that can move along the wire).

In addition, in the embodiment described above, the computers 50, 70,100, 150 and 200 have been described, but the technology of the presentdisclosure is not limited to theses. For example, instead of thecomputers 50, 70, 100, 150, and/or 200, a device including an ASIC, anFPGA, and/or a PLD may be applied. In addition, instead of the computers50, 70, 100, 150, and/or 200, a combination of a hardware configurationand a software configuration may be used.

In addition, in the embodiment described above, the display controldevice program is stored in the storage 60, but the technology of thepresent disclosure is not limited to this, and as shown in FIG. 47, forexample, the display control device program may be stored in anyportable storage medium 400, such as an SSD or a USB memory, which is anon-transitory storage medium. In this case, the display control deviceprogram stored in the storage medium 400 is installed in the computer50, and the CPU 58 executes the display control device side processaccording to the display control device program.

In addition, the display control device program may be stored in astorage unit of another computer or a server device connected to thecomputer 50 via a communication network (not shown), and the displaycontrol device program may be downloaded to the display control device12 in response to the request of the display control device 12. In thiscase, the display control device side process based on the downloadeddisplay control device program is executed by the CPU 58 of the computer50.

In addition, in the embodiment described above, the CPU 58 has beendescribed as an example, but the technology of the present disclosure isnot limited to this, and a GPU may be adopted. In addition, a pluralityof CPUs may be adopted instead of the CPU 58. That is, the displaycontrol device side process may be executed by one processor or aplurality of physically separated processors.

The following various processors can be used as a hardware resource forexecuting the display control device side process. Examples of theprocessor include a CPU, which is a general-purpose processor thatfunctions as software, that is, the hardware resource for executing thedisplay control device side process according to the program, asdescribed above. In addition, another example of the processor includesa dedicated electric circuit which is a processor having a circuitconfiguration specially designed for executing a specific process, suchas an FPGA, a PLD, or an ASIC. A memory is also built in or connected toeach processor, and each processor executes the display control deviceside process by using the memory.

The hardware resource for executing the display control device sideprocess may be configured by one of the various processors, or may be acombination of two or more processors of the same type or differenttypes (for example, a combination of a plurality of FPGAs or acombination of a CPU and an FPGA). In addition, the hardware resourcefor executing the display control device side process may be oneprocessor.

As an example of configuring the hardware resource with one processor,first, as represented by a computer such as a client computer or aserver, there is a form in which one processor is configured by acombination of one or more CPUs and software, and the processorfunctions as the hardware resource for executing the display controldevice side process. Secondly, as represented by SoC, there is an aspectin which a processor that realizes the functions of the whole systemincluding a plurality of the hardware resources for executing thedisplay control device side process with one IC chip is used. In thisway, the display control device side process is realized by using one ormore of the various processors described above as the hardware resource.

Further, as the hardware structure of these various processors, morespecifically, an electric circuit in which circuit elements such assemiconductor elements are combined can be used.

In addition, the display control device side process described above ismerely an example. Therefore, it is needless to say that unnecessarysteps may be deleted, new steps may be added, or the process order maybe changed within a range that does not deviate from the gist.

The contents described and shown above are the detailed description ofthe parts according to the technology of the present disclosure, and aremerely examples of the technology of the present disclosure. Forexample, the description of the configuration, the function, the action,and the effect above are the description of examples of theconfiguration, the function, the action, and the effect of the partsaccording to the technology of the present disclosure. Accordingly, itis needless to say that unnecessary parts may be deleted, new elementsmay be added, or replacements may be made with respect to the contentsdescribed and shown above within a range that does not deviate from thegist of the technology of the present disclosure. In addition, in orderto avoid complications and facilitate understanding of the partsaccording to the technology of the present disclosure, in the contentsdescribed and shown above, the description of common technologicalknowledge and the like that do not particularly require description forenabling the implementation of the technology of the present disclosureare omitted.

In the present specification, “A and/or B” is synonymous with “at leastone of A or B”. That is, “A and/or B” means that it may be only A, onlyB, or a combination of A and B. In addition, in the presentspecification, in a case in which three or more matters are associatedand expressed by “and/or”, the same concept as “A and/or B” is applied.

All of the documents, the patent applications, and the technicalstandards described in the present specification are incorporated in thepresent specification by referring to the same extent as a case in whichindividual document, patent application, and technical standard arespecifically and individually noted to be incorporated by reference.

Regarding the embodiment described above, the following supplementarynote will be further disclosed.

(Supplementary Note 1)

A display control device including a processor, and a memory built in orconnected to the processor, in which the processor acquires firstviewpoint position information indicating a first viewpoint position ofa first person with respect to an imaging region, performs a control ofdisplaying a first viewpoint video selected from among a plurality ofviewpoint videos generated based on images obtained by imaging theimaging region from a plurality of viewpoint positions different fromeach other on a first display unit, the first display unit being capableof displaying a video visually recognized by a second person differentfrom the first person, and performs a control of displaying firstspecific information for specifying the first viewpoint positionindicated by the acquired first viewpoint position information in thefirst viewpoint video and performs a control of changing a display sizeof the first specific information depending on an angle of view of thefirst viewpoint video displayed on the first display unit in a case inwhich the first viewpoint position indicated by the acquired firstviewpoint position information is included in the first viewpoint video.

What is claimed is:
 1. A display control device comprising: a processor;and a memory connected to or built in the processor, wherein theprocessor acquires first viewpoint position information indicating afirst viewpoint position of a first person with respect to an imagingregion, performs a control of causing a first display capable ofdisplaying a video visually recognized by a second person different fromthe first person to display a first viewpoint video selected from amonga plurality of viewpoint videos generated based on images obtained byimaging the imaging region from a plurality of viewpoint positionsdifferent from each other, and performs a control of displaying firstspecific information for specifying the first viewpoint positionindicated by the acquired first viewpoint position information in thefirst viewpoint video and performs a control of changing a display sizeof the first specific information depending on an angle of view of thefirst viewpoint video displayed on the first display in a case in whichthe first viewpoint position indicated by the acquired first viewpointposition information is included in the first viewpoint video.
 2. Thedisplay control device according to claim 1, wherein the processorperforms a control of setting a degree of difference between an imagequality of the first viewpoint video and an image quality of the firstspecific information within a first predetermined range.
 3. The displaycontrol device according to claim 1, wherein the processor performs acontrol of changing a display aspect of the first specific informationdepending on a relationship between a display size of the firstviewpoint video and the display size of the first specific information.4. The display control device according to claim 3, wherein theprocessor changes the display aspect of the first specific informationby performing a control of hiding the first specific information,performing a control of displaying only an outline of the first specificinformation, or performing a control of translucently displaying thefirst specific information in a case in which a ratio of the displaysize of the first specific information to the display size of the firstviewpoint video is equal to or more than a first threshold value.
 5. Thedisplay control device according to claim 3, wherein the processorchanges the display aspect of the first specific information dependingon the relationship between the display size of the first viewpointvideo and the display size of the first specific information and arelationship between a display position of the first viewpoint video anda display position of the first specific information.
 6. The displaycontrol device according to claim 3, wherein the processor performs acontrol of displaying the first specific information in a display aspectemphasized more than other regions in the first viewpoint video in acase in which a ratio of the display size of the first specificinformation to the display size of the first viewpoint video is lessthan a second threshold value.
 7. The display control device accordingto claim 1, wherein the first display is provided in a first headmounted display mounted on the second person.
 8. The display controldevice according to claim 1, wherein the first viewpoint video is aviewpoint video selected from among the plurality of viewpoint videos inresponse to a first instruction for selecting any one of the pluralityof viewpoint videos, which is received by a first reception devicecapable of receiving the first instruction.
 9. The display controldevice according to claim 1, wherein the processor further acquiresfirst visual line direction information indicating a first visual linedirection of the first person with respect to the imaging region, andthe first specific information includes information for specifying thefirst visual line direction indicated by the first visual line directioninformation acquired by the processor.
 10. The display control deviceaccording to claim 1, wherein each of the plurality of viewpoint videoshas peculiar viewpoint position information indicating a peculiarviewpoint position, each of the plurality of viewpoint videos is a videoshowing the imaging region observed from the corresponding peculiarviewpoint position, and the first viewpoint position information is thepeculiar viewpoint position information of any one of the plurality ofviewpoint videos.
 11. The display control device according to claim 10,wherein the processor acquires the peculiar viewpoint positioninformation corresponding to a second viewpoint video selected fromamong the plurality of viewpoint videos, which is displayed on a seconddisplay capable of displaying a video visually recognized by the firstperson and the second viewpoint video, as the first viewpoint positioninformation.
 12. The display control device according to claim 1,wherein the processor further acquires first visual line directioninformation indicating a first visual line direction of the first personwith respect to the imaging region, the first specific informationincludes information for specifying the first visual line directionindicated by the first visual line direction information acquired by theprocessor, and the processor acquires information indicating a directionfacing a second viewpoint video selected from among the plurality ofviewpoint videos, which is displayed on a second display capable ofdisplaying a video visually recognized by the first person and thesecond viewpoint video, as the first visual line direction information.13. The display control device according to claim 12, wherein the seconddisplay is provided in a second head mounted display mounted on thefirst person.
 14. The display control device according to claim 11,wherein the second viewpoint video is a viewpoint video selected fromamong the plurality of viewpoint videos in response to a secondinstruction for selecting any one of the plurality of viewpoint videos,which is received by a second reception device capable of receiving thesecond instruction.
 15. The display control device according to claim 1,wherein the processor acquires second viewpoint position informationindicating a second viewpoint position of the second person with respectto the imaging region, performs a control of causing a second displaycapable of displaying a video visually recognized by the first person todisplay a second viewpoint video selected from among the plurality ofviewpoint videos, and performs a control of displaying second specificinformation for specifying the second viewpoint position indicated bythe acquired second viewpoint position information in the secondviewpoint video and performs a control of changing a display size of thesecond specific information depending on an angle of view of the secondviewpoint video displayed on the second display in a case in which thesecond viewpoint position indicated by the acquired second viewpointposition information is included in the second viewpoint video.
 16. Thedisplay control device according to claim 15, wherein the processorperforms a control of setting a degree of difference between an imagequality of the second viewpoint video and an image quality of the secondspecific information within a second predetermined range.
 17. Thedisplay control device according to claim 15, wherein the processorperforms a control of changing a display aspect of the second specificinformation depending on a relationship between a display size of thesecond viewpoint video and the display size of the second specificinformation.
 18. The display control device according to claim 17,wherein the processor changes the display aspect of the second specificinformation by performing a control of hiding the second specificinformation, performing a control of displaying only an outline of thesecond specific information, or performing a control of translucentlydisplaying the second specific information in a case in which a ratio ofthe display size of the second specific information to the display sizeof the second viewpoint video is equal to or more than a third thresholdvalue.
 19. The display control device according to claim 15, wherein theprocessor changes the display aspect of the second specific informationdepending on a relationship between a display size of the secondviewpoint video and the display size of the second specific informationand a relationship between a display position of the second viewpointvideo and a display position of the second specific information.
 20. Thedisplay control device according to claim 17, wherein the processorperforms a control of displaying the second specific information in adisplay aspect emphasized more than other regions in the secondviewpoint video in a case in which a ratio of the display size of thesecond specific information to the display size of the second viewpointvideo is less than a fourth threshold value.
 21. The display controldevice according to claim 15, wherein the processor further acquiressecond visual line direction information indicating a second visual linedirection of the second person with respect to the imaging region, andthe second specific information includes information for specifying thesecond visual line direction indicated by the second visual linedirection information acquired by the processor.
 22. The display controldevice according to claim 15, wherein each of the plurality of viewpointvideos has peculiar viewpoint position information indicating a peculiarviewpoint position, each of the plurality of viewpoint videos is a videoshowing the imaging region observed from the corresponding peculiarviewpoint position, and each of the first viewpoint position informationand the second viewpoint position information is the peculiar viewpointposition information of any one of the plurality of viewpoint videos.23. The display control device according to claim 22, wherein theprocessor acquires the peculiar viewpoint position informationcorresponding to the second viewpoint video selected from among theplurality of viewpoint videos, which is displayed on the second displaycapable of displaying the video visually recognized by the first personand the second viewpoint video, as the first viewpoint positioninformation, and the processor acquires the peculiar viewpoint positioninformation corresponding to the first viewpoint video displayed on thefirst display as the second viewpoint position information.
 24. Thedisplay control device according to claim 15, wherein the processorfurther acquires first visual line direction information indicating afirst visual line direction of the first person with respect to theimaging region, the first specific information includes information forspecifying the first visual line direction indicated by the first visualline direction information acquired by the processor, the processoracquires information indicating a direction facing the second viewpointvideo displayed on the second display as the first visual line directioninformation, the processor further acquires second visual line directioninformation indicating a second visual line direction of the secondperson with respect to the imaging region, the second specificinformation includes information for specifying the second visual linedirection indicated by the second visual line direction informationacquired by the processor, and the processor acquires informationindicating a direction facing the first viewpoint video displayed on thefirst display as the second visual line direction information.
 25. Thedisplay control device according to claim 15, wherein the second displayis provided in a second head mounted display mounted on the firstperson.
 26. The display control device according to claim 15, whereinthe second viewpoint video is a viewpoint video selected from among theplurality of viewpoint videos in response to a second instruction forselecting any one of the plurality of viewpoint videos, which isreceived by a second reception device capable of receiving the secondinstruction.
 27. The display control device according to claim 15,wherein the processor performs setting for hiding the second specificinformation in a case in which a first predetermined condition issatisfied.
 28. The display control device according to claim 1, whereina viewpoint position of at least one of the first person or the secondperson with respect to the imaging region is limited to a partial regionof the imaging region.
 29. The display control device according to claim1, wherein the processor performs setting for hiding the first specificinformation in a case in which a second predetermined condition issatisfied.
 30. The display control device according to claim 1, whereinat least one of the plurality of viewpoint videos is a virtual viewpointvideo.
 31. A display control method comprising: acquiring firstviewpoint position information indicating a first viewpoint position ofa first person with respect to an imaging region; performing a controlof causing a first display capable of displaying a video visuallyrecognized by a second person different from the first person to displaya first viewpoint video selected from among a plurality of viewpointvideos generated based on images obtained by imaging the imaging regionfrom a plurality of viewpoint positions different from each other; andperforming a control of displaying first specific information forspecifying the first viewpoint position indicated by the acquired firstviewpoint position information in the first viewpoint video andperforming a control of changing a display size of the first specificinformation depending on an angle of view of the first viewpoint videodisplayed on the first display in a case in which the first viewpointposition indicated by the acquired first viewpoint position informationis included in the first viewpoint video.
 32. A non-transitorycomputer-readable storage medium storing a program for causing acomputer to execute a process comprising: acquiring first viewpointposition information indicating a first viewpoint position of a firstperson with respect to an imaging region; performing a control ofcausing a first display capable of displaying a video visuallyrecognized by a second person different from the first person to displaya first viewpoint video selected from among a plurality of viewpointvideos generated based on images obtained by imaging the imaging regionfrom a plurality of viewpoint positions different from each other; andperforming a control of displaying first specific information forspecifying the first viewpoint position indicated by the acquired firstviewpoint position information in the first viewpoint video andperforming a control of changing a display size of the first specificinformation depending on an angle of view of the first viewpoint videodisplayed on the first display in a case in which the first viewpointposition indicated by the acquired first viewpoint position informationis included in the first viewpoint video.